OLD Add Your Own Packages to the desc python Environment

If you are new to DESC and DESC python environments, please consult our related documentation for NERSC or IN2P3.

You may have additional packages or an updated version of a package that you want to install alongside desc-python.

• If this is a package of general interest, please consult within your Working Groups and then request the package be added to desc-python by opening an issue in desc-help.

If you would rather install the package(s) in your own area:

NOTE: After following the instructions here, to use your new environment in jupyter.nersc.gov, see below

• Create a full clone of the desc environment in your own area

  •  source /global/common/software/lsst/common/miniconda/setup_current_python.sh
     conda create --clone desc -p <PathToYourDir>/mydesc
    

  • Activate your new environment and add your desired packages

       conda activate <PathToYourDir>/mydesc
       conda install <Your Package> OR pip install <Your Package> 
    

Here is a full concrete example at NERSC

source /global/common/software/lsst/common/miniconda/setup_current_python.sh
conda create --clone desc -p /global/common/software/lsst/users/heatherk/mydesc
conda activate /global/common/software/lsst/users/heatherk/mydesc
conda install -c conda-forge firecrown

Now I have firecrown install in my own mydesc conda environment. To give it a try, I need the examples available in the firecrown git repo, so I clone it into my own area:

cd /global/common/software/lsst/users/heatherk
git clone https://github.com/LSSTDESC/firecrown

Then I try a simple example:

cd firecrown/examples/des_y1_3x2pt
firecrown compute des_y1_3x2pt.yaml
Watch out! Here comes a firecrown!
analysis id: 190918deca4946d8879ac0b3c3d19232
loglike: -236.47695322891724

A similar example at CC:

/pbs/home/h/hkelly(0)>source /pbs/throng/lsst/software/desc/common/miniconda/setup_current_python.sh
(desc) /pbs/home/h/hkelly(0)>conda create --clone desc --prefix=$PWD/mydesc
(desc) /pbs/home/h/hkelly(0)>conda activate /pbs/home/h/hkelly/mydesc
(/pbs/home/h/hkelly/mydesc) /pbs/home/h/hkelly(0)>pip install git+https://github.com/LSSTDESC/firecrown.git

Now I have firecrown install in my own mydesc conda environment. To give it a try, I need the examples available in the firecrown git repo, so I clone it into my own area:

git clone https://github.com/LSSTDESC/firecrown

Then I try a simple example, first starting up an interactive session on a compute node:

qlogin -l sps=1,s_fsize=1G,s_cpu=1:00:00,s_rss=1G
source /pbs/throng/lsst/software/desc/common/miniconda/setup_current_python.sh
conda activate /pbs/home/h/hkelly/mydesc
cd firecrown/examples/des_y1_3x2pt
firecrown compute des_y1_3x2pt.yaml
Watch out! Here comes a firecrown!
analysis id: 190918deca4946d8879ac0b3c3d19232
loglike: -236.47695322891724

Developer mode (not thoroughly checked)

If a package is included in a cloned env, the logical steps would be

conda uninstall <package> --force
git clone <package>
cd <package> && pip install --no-deps -e .

Jupyter At NERSC

Once you have your conda environment installed, you can use this environment in jupyter.nersc.gov by setting a new environment variable in your NERSC $HOME/.bashrc or $HOME/.cshrc. If using the bash shell, then you would add this to your $HOME/.bashrc (create this file if it does not already exist):

export DESCUSERENV=<FullPathToYourCondaEnv>

Example:

export DESCUSERENV=/global/common/software/lsst/users/heatherk/mydesc

With the DESCUSERENV environment variable set, the next time you start the desc-python jupyter kernel, you will be using your own environment, rather than the official desc-python.

NOTE: If you have previously set DESCPYTHONPATH, it is strongly recommended that you remove the DESCPYTHONPATH environment variable from your $HOME/.bashrc (or $HOME/.bashrc.ext). This feature is currently deprecated as we see how the new DESCUSERENV works out.

Docker Coming Soon...