Page tree

The license could not be verified: License Certificate has expired!

 What is Kita Lab?

The lab is an interactive development environment for working notebooks, code, and data (especially HDF data).

 What browsers are supported?

The lab works with most browsers, but is most thoroughly tested with Chome and Firefox.

 Where is Kita Lab Hosted?

The lab is hosted on Amazon AWS, in the us-west-2 region (Oregon, USA)

 How is Kita Lab different than JupyterLab?

The lab is based on the Open Source JupyterLab project, but contains several extensions to facilitate working with HDF data:

  • Lab users are configured with access to a co-located instance of Kita Server. The Kita Server provides efficient access to large HDF Collections
  • In addition to standard packages for data analytics: pandas, xarray, bokeh, dask, etc. the kernel has packages for working with HDF data, either HDF5 files on the local disk (h5py), or content on Kita Server (h5pyd).
  • Command line tools are provided for working with either HDF data. Either local HDF5 files (h5dump, h5ls, h5repack, etc.) or HDF content on the server (hsinfo, hsls, hsget, etc.)
  • Sample data is provided with HDF server in the folder: hdf5://shared/.
  • Example notebooks are provided: /home/jovyan/examples
 Is it free to use?

Everyone gets a 30-day free trial of Kita Lab. After the free trial, there is a small subscription fee.

 What resources do I have access to?

Your account gives you:

  • 8GB RAM for running your Jupyter Notebooks
  • A dedicated Xeon CPU core
  • 10GB local disk
  • 200GB of storage on Kita Server
 Where is my data stored?

Your 10GB disk is stored as an AWS EBS Volume. Content in Kita Server is stored in a S3 Bucket. Each user has a server folder where they have permissions to write to. The name of the folder is listed in the output of the "hsinfo" command (see "command line tools" below). Typically it is
"/home/myname" where "myname" is your login name prior to the "@" symbol. In addition there is a server folder names "/shared" that contains data files accessible to everyone.

 Can I import existing Jupyter/IPython notebooks?

Yes, click on the "upload" button in the UI () and select the file you wish to upload.

 How do I copy and paste from the Kita Lab terminal?

Use Shift+Right Click (Ctrl+Shift on Mac) and select copy from the popup menu.

 Are there samples and tutorials?

Yes, in the directory "/home/jovyan/examples" you will find sample notebooks and a tutorial. Running the different notebooks in the tutorial is a good way to become familiar with the capabilities of Kita Server. The GitHub repository for the examples is located at https://github.com/HDFGroup/hdflab_examples.

 Can I share my notebooks with other users?

The easiest method would be to create a Github repository and have other users clone the repo.

 Can I import my HDF5 files to Kita Server?

Yes, follow these steps:

  • Upload the file to the lab using the upload button
  • Upon a terminal window (click '+' and then select terminal from the launcher)
  • Run the command: $ hsinfo , this will display information about the Kita Server, including your "home" folder, e.g. /home/myname/
  • Run the command: $ hsload myfile.h5 /home/myname/
  • Run the command: $ hsls /home/myname/myfile.h5 to verify the contents get stored on Kita
  • Server
 I have a file larger than 10GB. How can I upload it to Kita Server?

You can upload directly from your desktop. Follow these steps:

  • Install h5pyd on your desktop: $ pip install h5pyd
  • Download the file /home/jovyan/.hscfg to the home directory on your desktop
  • Run hsload as above
 Can I share files on my local disk with other users?

 No, not directly, though you could share via git or other such means.

 Can I share content on Kita Server with other users?

 Yes, use the hsacl tool to make your folder or domain publically readable. If the domain is: /home/myname/mydata.h5, run the command: $ hsacl /home/myname/mydata.h5 +r default .

To enable specific users to read the folder or domain, replace "default" in the above command with the user's email address that they use when logging in to Kita Lab.

You can also enable other users to have write permission to the domain, see the hsacl help info.

 If I run into a problem or have additional questions, where can I go for help?

 You can get free help from the HDF community at https://forum.hdfgroup.org.

 What are the command line tools provided for accessing Kita Server?

 These are listed below.

hsinfo

This tool returns basic information on the server state and validates the username and password saved in the .hscfg file. For example:

$ hsinfo
endpoint: http://hsdshdflab.hdfgroup.org
server name: HDF Server for Kita Lab
server state: READY
username: test_user1
password: ********************************
home: /home/test_user1/
server version: 0.3
up: 55 min 27 sec
h5pyd version: 0.3.3

If you add a domain argument to hsinfo, summary information about that domain will be displayed. Example:

$ hsinfo /shared/tall.h5
domain: /shared/tall.h5
owner: admin
id: g-75b88d05-46db146e-fd30-058dce-539ce6
last modified: 2019-01-17 15:39:36
total_size: 6149
allocated_bytes: 580
num objects: 10
num chunks: 4

hsls

This tool list either folders and domains at the given path.

Rather than directories and files, Kita uses the terms folders and domains. A folder is used to store collections of folders and domains. A domain is the equivalent to a regular HDF5 file stored in a POSIX filesystem.

During installation, a home folder is created for each user in the path: /home/<username> . If peterpan is a username, running $hsls /home/peterpan/ will display:

peterpan                     folder           2018-06-25    17:01:24     /home/peterpan/
1 items

As additional content is created in this folder, those items will be displayed by hsls.

hsload

This tool will upload an HDF5 from the client environment to Kita.

Usage: $ hsload <filename> <folder> to upload a file to the given folder (where it will be saved using the same name as the local file), or $hsload <filename> <domain> , to store to the given domain.

Examples:

hsload myhdf5file.h5 /home/bob/                  # creates
/home/bob/myhdf5file.h5
hsload myhdf5file.h5 /home/bob/mycloudfile.h5 # creates
/home/bob/mycloudfile.h5

Note: There's no equivalent to the concept of "current working directory" for paths on the server. Always use absolute path names.

Note: By default hsload doesn't produce any output. Use the -v flag or use the hsls tool to verify the load.

Note: The requesting user most have permissions to write to the folder that will be used to store the domain (see hsacl below).

Note: Non-HDF5 files cannot be loaded. Use h5ls to verify that a file to uploaded is indeed an HDF5 file.

Note: If you need a test file, you can download this one: https://s3.amazonaws.com/hdfgroup/data/hdf5test/tall.h5.

hsget

This tool retrieves a domain from the server an stores it as a ordinary HDF5 file on the client system.

Usage: $ hsget <domain> <filename>

Examples:

hsget /home/bob/myhdf5file.h5 myhdf5file.h5 # stores myhdf5file.h5 in the
current directory

hstouch

This tool creates a new folder or a domain (which will contain just the root group) on the server.

Usage: $ hstouch <folder> to create a folder

Usage: $ hstouch <domain> to create a domain

Example:

hstouch /home/bob/myfolder

hsrm

This tool removes a folder or domain from the server.

Usage: $ hsrm <folder> to delete a folder

Usage: $ hsrm <domain> to delete a domain

Example:

hsrm /home/bob/myhdf5file.h5

hsmv

This tool can be used to move a domain from one folder location to another.

Usage: $ hsmv <src_domain> <des_domain> to move a domain

Usage: $ hsmv <src_domain> <des_folder> to move domain into folder des_folder

Example:

hsmv /home/bob/folder1/myhdf5file.h5 /home/bob/folder2/

hscp

This tool copies all content from the given domain to a new domain.

Usage: $ hscp <src_domain> <des_domain>

Example:

hscp /home/bob/myhdf5file.h5 /home/bob/myhdf5file_copy.h5

hsdiff

This tool compares the content of an HDF5 file with a domain and prints a message if the two objects are different.

Usage: $ hsdiff <HDF5 file> <domain>

Example:

$ hsdiff myhdf5file.h5 /home/bob/myhdf5file.h5

hsacl

This tools can be used to display or modify ACLs (Access Control List) for folders or domains. The ACL for a given user controls which actions ("create", "read", "update", "delete", "read ACL", "update ACL") can be performed (either via the CLI tools, or programmatically).

Run: hsacl --help for details on usage.

Also, this blog post: https://www.hdfgroup.org/2015/12/serve-protect-web-security-hdf5/ describes the design and use of ACLs as well as well as other web security related issues.

 

 

--- Last Modified: May 02, 2019 | 05:24 PM