BaseSpace File System

For a general overview of Native Apps, please refer to the Native App Overview.

What is BSFS?

BaseSpace FileSystem, also known as BaseSpace FS or BSFS, is a Fuse file system layer implemented over the BaseSpace API. It provides a virtual file system view on the data contained in BaseSpace's Samples and AppResults. BSFS is used in the Native App Engine to provide applications running in docker containers with access to the data the user specified in the input form through a read only mount point on the file system. Using BSFS, the data is only downloaded at the moment an application tries to access it through the file system on the docker container. It was designed to replace the lengthy download step, which precedes every Native Application launch. Using BSFS, the application can read portions of files without having to download the entire file beforehand, and this process is completely invisible to the developer.

BSFS features a pre-fetch mechanism, which is used to mitigate performance issues due to latency of network requests. Data is downloaded using blocksize units of size 5,000,000 bytes. Every read to the file system is rounded off to the closest block boundaries and downloaded before the read call returns the data. Concurrently to fetching the data in the range requested a pre-fetch is scheduled to read ahead the data in the block following the last requested block.

BSFS is the new default way to access your data in the Native App Engine; pre-download of the data by spacedock is now a deprecated mechanism and will not be supported going forward.

Advantages of using BSFS

  • A download step is no longer required to make all the input data available and your app can start running right away.
  • Your application can access only portions of samples or app results data without incurring the time cost of downloading the entire datasets.
  • An application can gain on execution time by overlapping computation and data transfers.
  • A multi-node application can now efficiently parallelize processing over a single input App Result or sample, without incurring the overhead of downloading the input entirely.
  • Provides the same view on your data as if it had been downloaded by spacedock in the pre-download step.

Using BSFS

The Native App Conventions page provides information on how to enable BSFS on your existing application if you created the application prior to November 2014.