The Data Movement Cancel Request permits users to initiate a cancellation of a data movement request that is in progress.
| Field | Type | Label | Description |
| workflow | DataMovementWorkflow | The name and namespace of the initiating workflow |
|
| uid | string | The unique identifier for the data movement resource |
The Data Movement Cancel Response returns the status of the cancel request. The cancel process will begin upon success. This response does not indicate that that cancel process has completed, but rather that it has been initiated.
| Field | Type | Label | Description |
| status | DataMovementCancelResponse.Status | Status of the Data Movement Cancellation Request |
|
| message | string | Any extra information included with the status |
Data movement operations contain a CommandStatus in the Status object. This information relays the current state of the data movement progress.
| Field | Type | Label | Description |
| command | string | Command used to perform Data Movement |
|
| progress | int32 | Progress percentage (0-100%) of the data movement based on the output of `dcp`. `dcp` must be present in the command. |
|
| elapsedTime | string | Duration of how long the operation has been running (e.g. 1m30s) |
|
| lastMessage | string | The most recent line of output from the data movement command |
|
| lastMessageTime | string | The time (local) of lastMessage |
Create a Data Movement Request to perform data movement. NNF Data Movement will copy from source to the destination path.
| Field | Type | Label | Description |
| workflow | DataMovementWorkflow | The name and namespace of the initiating workflow |
|
| source | string | Source file or directory |
|
| destination | string | Destination file or directory |
|
| dryrun | bool | If True, the data movement command runs `/bin/true` rather than perform actual data movement |
|
| dcpOptions | string | Extra options to pass to `dcp` if present in the Data Movement command. |
|
| logStdout | bool | If true, enable server-side logging of stdout when the command is successful. Failure output is always logged. |
|
| storeStdout | bool | If true, store stdout in DataMovementStatusResponse.Message when the command is successful. Failure output is always contained in the message. |
|
| slots | int32 | The number of slots specified in the MPI hostfile. A value of 0 disables the use of slots in the hostfile. -1 will defer to the server side configuration. |
|
| maxSlots | int32 | The number of max_slots specified in the MPI hostfile. A value of 0 disables the use of max_slots in the hostfile. -1 will defer to the server side configuration. |
|
| profile | string | The name of the data movement configuration profile to use. The above parameters (e.g. slots, logStdout) will override the settings defined by the profile. This profile must exist on the server otherwise the data movement operation will be invalid. Empty will default to the default profile. |
|
| mpirunOptions | string | Extra options to pass to `mpirun` if present in the Data Movement command. |
The Data Movement Create Response to indicate the status of of the Data Movement Request.
| Field | Type | Label | Description |
| uid | string | The unique identifier for the created data movement resource if `Status` is Success |
|
| status | DataMovementCreateResponse.Status | Status of the DataMovementCreateRequest |
|
| message | string | Any extra information supplied with the status |
The Data Movement Delete Request permits users to delete a completed data movement request (any request with a status of COMPLETED).
| Field | Type | Label | Description |
| workflow | DataMovementWorkflow | The name and namespace of the initiating workflow |
|
| uid | string | The unique identifier for the data movement resource |
The Data Movement Delete Response returns the status of the delete operation.
| Field | Type | Label | Description |
| status | DataMovementDeleteResponse.Status | Status of the Data Movement Delete Request |
|
| message | string | Any extra information supplied with the status |
The Data Movement List Request allows a user to get a list of the current data movement requests for a given DWS Workflow.
| Field | Type | Label | Description |
| workflow | DataMovementWorkflow | The name and namespace of the initiating workflow |
The Data Movement List Response returns a list of the matching data movement requests' UIDs
| Field | Type | Label | Description |
| uids | string | repeated | List of data movement requests associated with the given workflow and namespace |
The Data Movement Status Request message permits users to query the status of a Data Movement Request.
| Field | Type | Label | Description |
| workflow | DataMovementWorkflow | The name and namespace of the initiating workflow |
|
| uid | string | UID of the Data Movement Request |
|
| maxWaitTime | int64 | The maximum time in seconds to wait for completion of the data movement resource. Negative values imply an indefinite wait |
The Data Movement Status Response message defines the current status of a data movement request.
| Field | Type | Label | Description |
| state | DataMovementStatusResponse.State | Current state of the Data Movement Request |
|
| status | DataMovementStatusResponse.Status | Current status of the Data Movement Request |
|
| message | string | Any extra information supplied with the state/status |
|
| commandStatus | DataMovementCommandStatus | Current state/progress of the Data Movement command |
|
| startTime | string | The start time (local) of the data movement operation |
|
| endTime | string | The end time (local) of the data movement operation |
The data movement version response returns information on the running data movement daemon.
| Field | Type | Label | Description |
| version | string | Current version of the API. |
|
| apiVersions | string | repeated | List of supported API versions. |
Data Movement workflow message contains identifying information for the workflow initiating the data movement operation. This message must be nested in all Request messages.
| Field | Type | Label | Description |
| name | string | Name of the DWS workflow that is associated with this data movement request |
|
| namespace | string | Namespace of the DWS workflow that is associated with this data movement request |
| Name | Number | Description |
| INVALID | 0 | The cancel request was found to be invalid |
| NOT_FOUND | 1 | The request with the supplied UID was not found |
| SUCCESS | 2 | The data movement request was cancelled successfully |
| FAILED | 3 | The data movement request cannot be canceled |
| Name | Number | Description |
| SUCCESS | 0 | The data movement resource created successfully |
| FAILED | 1 | The data movement resource failed to create. See `message` field for more information |
| INVALID | 2 | Invalid request |
| Name | Number | Description |
| INVALID | 0 | The delete request was found to be invalid |
| NOT_FOUND | 1 | The request with the supplied UID was not found |
| SUCCESS | 2 | The data movement request was deleted successfully |
| ACTIVE | 3 | The data movement request is currently active and cannot be deleted |
| UNKNOWN | 4 | Unknown status |
| Name | Number | Description |
| PENDING | 0 | The request is created but has a pending status |
| STARTING | 1 | The request was created and is in the act of starting |
| RUNNING | 2 | The data movement request is actively running |
| COMPLETED | 3 | The data movement request has completed |
| CANCELLING | 4 | The data movement request has started the cancellation process |
| UNKNOWN_STATE | 5 | Unknown state |
| Name | Number | Description |
| INVALID | 0 | The request was found to be invalid. See `message` for details |
| NOT_FOUND | 1 | The request with the supplied UID was not found |
| SUCCESS | 2 | The request completed with success |
| FAILED | 3 | The request failed. See `message` for details |
| CANCELLED | 4 | The request was cancelled |
| UNKNOWN_STATUS | 5 | Unknown status |
DataMover service definition describes the API for perform data movement for NNF storage.
| Method Name | Request Type | Response Type | Description |
| Version | .google.protobuf.Empty | DataMovementVersionResponse | Version sends a request to the data movement daemon and returns a response containing details on the current build version and supported API versions. |
| Create | DataMovementCreateRequest | DataMovementCreateResponse | Create sends a new data movement request identified by source and destination fields. It returns a response containing a unique identifier which can be used to query the status of the request. |
| Status | DataMovementStatusRequest | DataMovementStatusResponse | Status requests the status of a previously submitted data movement request. It accepts a unique identifier that identifies the request and returns a status message. |
| Delete | DataMovementDeleteRequest | DataMovementDeleteResponse | Delete will attempt to delete a completed data movement request. It accepts a unique identifier that identifies the request and returns the status of the delete operation. |
| List | DataMovementListRequest | DataMovementListResponse | List returns all current data movement requests for a given namespace and workflow. |
| Cancel | DataMovementCancelRequest | DataMovementCancelResponse | Cancel will attempt to stop a data movement request. It accepts a unique identifier that identifies the request and returns the status of the cancel operation. |
| .proto Type | Notes | C++ | Java | Python | Go | C# | PHP | Ruby |
| double | double | double | float | float64 | double | float | Float | |
| float | float | float | float | float32 | float | float | Float | |
| int32 | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required) |
| int64 | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. | int64 | long | int/long | int64 | long | integer/string | Bignum |
| uint32 | Uses variable-length encoding. | uint32 | int | int/long | uint32 | uint | integer | Bignum or Fixnum (as required) |
| uint64 | Uses variable-length encoding. | uint64 | long | int/long | uint64 | ulong | integer/string | Bignum or Fixnum (as required) |
| sint32 | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required) |
| sint64 | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. | int64 | long | int/long | int64 | long | integer/string | Bignum |
| fixed32 | Always four bytes. More efficient than uint32 if values are often greater than 2^28. | uint32 | int | int | uint32 | uint | integer | Bignum or Fixnum (as required) |
| fixed64 | Always eight bytes. More efficient than uint64 if values are often greater than 2^56. | uint64 | long | int/long | uint64 | ulong | integer/string | Bignum |
| sfixed32 | Always four bytes. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required) |
| sfixed64 | Always eight bytes. | int64 | long | int/long | int64 | long | integer/string | Bignum |
| bool | bool | boolean | boolean | bool | bool | boolean | TrueClass/FalseClass | |
| string | A string must always contain UTF-8 encoded or 7-bit ASCII text. | string | String | str/unicode | string | string | string | String (UTF-8) |
| bytes | May contain any arbitrary sequence of bytes. | string | ByteString | str | []byte | ByteString | string | String (ASCII-8BIT) |