FTP to AWS S3 Service (FTP2S3)
FTP tp AWS S3 is the first stage of the new data management service implementation. It is represented as FTP access to
AWS Simple Storage Service (S3), a service that allows you to store any amounts of your data in Cloud, and share it with your colleagues.
By default, storage management is performed via AWS Management Console and AWS CLI.
Establishing FTP access to AWS S3 storage allows to bring data management to a more common format. In addition, people who have access to S3 via FTP, do not have to have AWS credentials
to work with the storage, which increases the project security.
The service is available in AWS regions only.
For more information on AWS S3 service, visit the
Related CLI Commands
The table below provides the list of service-related commands and their descriptions:
|or2-manage-service...-s backup -a -k key_name
||Activates the service in the specified project and AWS region
|or2-manage-ftp...-a [mount, list, unmount] -b bucket_name
||Manages S3 buckets in the specified project and AWS region
|or2-ftp-access... -a [grant, list, revoke] -e user_email
||Manages user access to the FTP server
|or2-describe-instances -S backup
||Displays the details of VMs created during the service activation
|or2-describe-services -s backup
||Describes the service activated in the specified project and region
Further on this page, you can find more details on the FTP to AWS S3 service manipulation.
Pre-Requisites and Limitations
The service can be described as a medium between users and AWS S3. For that reason, there is a number of pre-requisites to be met for the successful service usage:
The project should be activated in AWS and have S3 service in use
All bucket management operations - creating, removing and renaming buckets - should be performed by native AWS tools (Management Console and CLI)
The user to have the FTP access has to have the %maestro_cli%lib/default.cr file, created by their or2access command call
FTP2S3 Servcie Activation and Manipulation
To athe service, use the or2-manage-service (or2ms) command with the -a/--activate flag and the -s/--service-name parameter with backup value.
The -k key_name parameter is also necessary, as for any instance in AWS:
or2ms -p project -r aws-region -a -s backup -k key_name
By default, FTP servers are SMALL-shaped VMs with Ubuntu14.04_64-bit operating system.
It is recommended to start the service in the region, in which the project S3 buckets are stored.
As soon as the service is activated, its data can be retrieved using the or2-describe-services (or2dser) command:
or2dser -p project -r region -s backup
You can find the details on the VMs created within the service activation, by calling the or2-describe-instances (or2din) command with -S backup parameter:
or2din -p project -r region -S backup
To deactivate the service, run the or2ms command with the -d/--deactivate flag:
or2ms -p project -r aws-region -d -s backup
User Access Management
By default, users cannot access the FTP server. For each user, the access should be granted explicitly. User access management is performed by means of the or2-ftp-access (or2ftpa) command:
To grant access to a user, run the command with the -a grant action:
or2ftpa -p project -r aws_region -a grant -e firstname.lastname@example.org
To see the list of the users to whom access is granted, run the command with the -a list action:
or2ftpa -p project -r aws_region -a list
To prohibit access to project S3 via FTP for a user to whom the access was granted previously, use the -a revoke action:
or2ftpa -p project -r aws_region -a revoke -e email@example.com
The users who have access to FTP can read, upload and delete from the buckets.
S3 Buckets Management
By default, the service does not provide access to any S3 buckets. They need to be registered, or mounted, on the FTP server.
If necessary, the bucket can be deactivated on the server, so that FTP access to it gets restricted. These manipulations are performed with the or2-manage-ftp (or2ftp) command.
The command references bucket names that must correspond to the AWS naming standards and be the same as the name specified for the target AWS S3 bucket:
To register a new S3 bucket, run the command with the -a mount action. The bucket name must correspond to the AWS naming standards and be the same as the name specified for the target AWS S3 bucket:
or2ftp -p project -r aws_region -a mount -b demo-ftp-to-s3-bucket
To add several buckets, repeat the command.
To view the list of the buckets registered on the FTP server, run the command with the -a list action:
or2ftp -p project -r aws_region -a list
To restrict FTP access to an S3 bucket, use the -a unmount parameter:
or2ftp -p project -r aws_region -a unmount -b demo-ftp-to-s3-bucket
Please note that this action does not block or restrict the bucket itself. It will still be available by native AWS tools.
Accessing the FTP Server
The users can access the FTP server by any FTP client convenient for them. The following login details should be specified:
||DNS or Public IP
||Token generated by Orchestrator at or2access command run and stored in the default.cr file
You can find the server DNS and Port in the webUiURL column of the or2-describe-services (or2dser) command response:
When connected, the user can start working with the S3 storage. The buckets will be given as top-level directories:
Within each bucket, the users can create, change, and delete the necessary files.
The price of the service consists of the price for the FTP Server instance (SMALL, Ubuntu14.04_64-bit), the price of the S3 service used, and the price for additional services, such as data transfer.
Each of these points depends on AWS pricing policy in each specific region.
EPAM Cloud does not apply any additional costs.