This section describes how user can move objects between S3 buckets using Lyftrondata.
Prerequisite
- Access to Lyftrondata Workflows.
- S3.exe provided by Lyftrondata.
S3.exe
S3.exe is a command line tool developed by Lyftrondata to interact with AWS S3.
Following are the commands currently supported by this tool.
Command Operations
|
Key |
Value |
|
Operations |
Downloadlatest, MoveObjects |
Command Arguments
|
Key |
Value |
|
-h |
Print help message. |
|
-credentials |
AccessKey and Secretkey to use when accessing the api. Values are delimited by colon. |
|
-region |
Aws Region to use when accessing the api. For Example us-east-1. |
|
-src |
Aws S3 bucket to read files or local directory to read files. |
|
-out |
Filename or bucket with full path to save the output data. |
|
-fileextension |
File extension to read from bucket. For Example zip or csv only one at a time |
Command Tasks
|
Key |
Value |
|
-unzip |
Unzip files with extension .zip and delete unzipped file. |
Example
S3 MoveObjects -credentials awsAccessKey:awsSecretKey -region us-west-2 -fileextension zip -src lyftrondata/Pearson -out lyftrondata/Pearson/Unzipped -unzip
Process
Enter workflow name and click Create.
Click on the workflow.
Click Steps
Click Add Step.
Choose Step Type as Stored Procedure and Job as lyft_execute_os_command.
Add path to executable (usually C:\S3.exe placed on the server) then add command and click create button.
Click START WORKFLOW to start your workflow.
Loom Video Link