Pydio v8

Cells Client

Cells Client provides an easy way to communicate with a Pydio Cells server instance from the command line (or from automation scripts).

Overview

This guide explains how to install, configure and use the tool on Linux, please adapt if you are using another operating system.

Cells Client a.k.a cec works like standard command line tools like ls, scp, etc. Using the cec command, you can list, download and upload directly to your remote Cells server.

Installation

Cells Client is a single self-contained binary file and is easy to install.

1 - Download cec

Grab the built version for your corresponding amd64 architectures:

2 - Make it executable

Give execution permissions to the binary file, typically on Linux: chmod u+x cec.

3 - Add it to the PATH (optional)

Add the command to your PATH environment variable, to makes it easy to call the command from anywhere in the system. On Linux, you can for instance add a symlink to the binary location (replace below with correct path):

sudo ln -s /path/to/your/binary/cec /usr/local/bin/cec

4 - Check for correct installation

To verify that cec is correctly installed, simply run for instance:

$ cec version
# Should output something like below
Cells Client
 Version:       2.1.0-rc1
 Built:         03 Mar 21 16:26 +0000
 Git commit:    4d09aa8e33fc60e65625e9f8435fd90b99c1b801
 OS/Arch:       linux/amd64
 Go version:    go1.15.5

Connecting To Cells

Cells Client requires an authenticated connection to the target Cells server. For a given user, read/write permissions are applied in the same way as what you see in the web interface.

Once a valid user is available, there are 2 options:

  • Go through an interactive configuration and persist necessary information on the client machine (Persistent Mode)
  • Pass the necessary connection information at each call (Non Persistent Mode)

Persistent Mode

Connection can be configured and persisted locally on the client machine.

Calling the cec configure command offers various authentication mechanisms. For persistent mode, we advise to use the default OAuth Authorization Code flow.

cec configure oauth

You will be guided through a few steps to configure and persist your connection:

  • Enter your server address: the full URL to access your Cells instance, e.g.: https://files.example.com/
  • Choose OAuth2 process either by opening a browser or copy/pasting the URL in your browser to get a valid token
  • Test and validate the connection.

The token is saved locally and will be refreshed automatically as required. If a keyring mechanism is available on the machine, it used to store sensitive information. You can verify this with the following command:

cec configure check-keyring

Supported keyrings are MacOSX Keychain, Linux DBUS and Windows Credential Manager API.

Non Persistent Mode

This mode can be useful to use the Cells Client in a CI/CD pipe or via cron jobs. In such case, we strongly advise you to create a Personal Access Token on the server and use this.

To create a token that is valid for user robot for 90 days, log via SSH into your server as pydio (a.k.a. as the user that runs the cells service) and execute:

$ cells admin user token -u robot -e 90d
✔ This token for robot will expire on Tuesday, 01-Jun-21 16:46:40 CEST.
✔ d-_-x3N8jg9VYegwf5KpKFTlYnQIzCrvbXHzS24uB7k.mibFBN2bGy3TUVzJvcrnUlI9UuM3-kzB1OekrPLLd4U
⚠ Make sure to secure it as it grants access to the user resources!

Then use environment variables (or the corresponding command flags) to pass connection information:

export CEC_URL=https://files.example.com
export CEC_TOKEN=d-_-x3N8jg9VYegwf5KpKFTlYnQIzCrvbXHzS24uB7k.mibFBN2bGy3TUVzJvcrnUlI9UuM3-kzB1OekrPLLd4U

Now you can directly talk to your server, for instance:

cec ls common-files

Usage

Use the cec --help command to know about available commands. Below are a few interesting ones for manipulating files:

  • cec ls: List files and folders on the server, when no path is provided, it lists the workspaces that the current user can access.
  • cec scp: Upload/Download file to/from a remote server.
  • cec cp, cec mv and cec rm: Copy, move, rename and delete files within the server.
  • cec mkdir: Create a folder on the remote server
  • cec clear: Clear authentication tokens stored in your keychain.

For your convenience, below are a few examples.

1/ Listing the content of the personal-files workspace

$ cec ls personal-files
+--------+--------------------------+
|  TYPE  |           NAME           |
+--------+--------------------------+
| Folder | personal-files           |
| File   | Huge Photo-1.jpg         |
| File   | Huge Photo.jpg           |
| File   | IMG_9723.JPG             |
| File   | P5021040.jpg             |
| Folder | UPLOAD                   |
| File   | anothercopy              |
| File   | cec22                    |
| Folder | recycle_bin              |
| File   | test_crud-1545206681.txt |
| File   | test_crud-1545206846.txt |
| File   | test_file2.txt           |
+--------+--------------------------+

2/ Showing details about a file

$ cec ls personal-files/P5021040.jpg -d
Listing: 1 results for personal-files/P5021040.jpg
+------+--------------------------------------+-----------------------------+--------+------------+
| TYPE |                 UUID                 |            NAME             |  SIZE  |  MODIFIED  |
+------+--------------------------------------+-----------------------------+--------+------------+
| File | 98bbd86c-acb9-4b56-a6f3-837609155ba6 | personal-files/P5021040.jpg | 3.1 MB | 5 days ago |
+------+--------------------------------------+-----------------------------+--------+------------+

3/ Uploading a file to server

$ cec scp ./README.md cells://common-files/
Copying ./README.md to cells://common-files/
 ## Waiting for file to be indexed...
 ## File correctly indexed

4/ Download a file from server

$ cec scp cells://personal-files/IMG_9723.JPG ./
Copying cells://personal-files/IMG_9723.JPG to ./
Written 822601 bytes to file

Appendix 1 - Authentication Modes Pros and Cons

There are 3 authentication methods to establish the connection:

Personal Access Token

A token can be generated on the server for a given user. It can be limited in time or you might choose the auto-refresh mode.

Note about the autorefresh option: let's say you have given a validity of 10 days. If you connect to your server within 10 days, the token's validity is extended for 10 more days on the server side: on the client side the token string remains unchanged.
Thus, if you have a cron job that runs once a week during the night, typically to push some backups to your server, the token remains valid undefinitly. But if your server is down and misses a week, the upload will fail the week after, because 14 days have passed and the token expired.

Pros:

  • Secure
  • Non Interactive
  • The best solution if the client machine is a headless server, that has no Keyring and must communicate with your server with daemon processes, typically cron jobs.
  • On client side, you only have to give the URL and the token to establish the connection

Cons:

  • To created a token, you must either have access to the server as privileged user or ask your sysadmin

OAuth2 Credential Flows

This is the recommended strategy for persitent mode on your local workstation. Calling cec configure oauth will guide you through a quick process to securely generate an ID token and a refresh Token.

Under the hood, cec will watch the validity of the token. When necessary, it will issue a refresh request and stores the updated tokens in your keychain without you even noticing it. For the record, in Cells 2.2, the default validity period of the refresh token is 60 days.

As tokens are represented as unique (random) complicated strings, this approach makes it difficult to steal your token by only looking at it, even if it ends up in clear text shown to third persons.

Pros:

  • Secure
  • Any user can use her own account to configure a connection, without asking the sysadmin.

Cons:

  • You must go through an interactive process configure your connection.

Client Credential Flows

This legacy method is not recommended and might disapear in a future version.

Pros:

  • You only have to enter URL, login and password
  • Can be used via configure process or directly using flags / env variable at each call
  • Any user can use her own account to configure a connection, without asking the sysadmin.

Cons:

  • The user password ends up stored in clear text in case no keyring is present
  • The process will fail if your server relies on external user repository to manage authentication (typically LDAP or SSO).

Appendix 2 - Command Completion

Cells Client provides a handy feature that provides completion on commands and paths; both on local and remote machines.

NOTE: you must add cec to you local PATH if you want to configure the completion helper (see above).

Bash completion

To enable this feature, you must have bash-completion third party add-on installed on your workstation.

# on Debian / Ubuntu
sudo apt install bash-completion

# on RHEL / CentOS
sudo yum install bash-completion

# on MacOS (make sure to follow the instructions displayed by Homebrew)
brew install bash-completion

MacOS latest release changed the default shell to ZSH.

Then, to add the completion in a persistent manner:

# Linux users
cec completion bash | sudo tee /etc/bash_completion.d/cec
# MacOS users 
cec completion bash | sudo tee /usr/local/etc/bash_completion.d/cec

You can also only source the file in current session, the feature will be gone when you start a new shell.

source <(cec completion bash)

Note: if you want to use completion for remote paths while using scp sub command, you have to prefix the remote path with cells// rather than cells://; that is to omit the column character before the double slash. Typically:

cec scp ./README.md cells//com <press the tab key>
# Completes the path to
cec scp ./README.md cells//common-files/
...

Note: when you update the Cells Client, you also have to update the completion file, typically on Linux machines:

cec completion bash | sudo tee /etc/bash_completion.d/cec
source /etc/bash_completion.d/cec
Back to top