Cannot install full shotgun databases (wmgx) in the biobakery VM image

[hyfood]

Dear developer(s),

I am new to the biobakery VM images and currently I am going through the biobakery_workflows tutorials following:

https://github.com/biobakery/biobakery/wiki/biobakery_workflows

However, I found it difficult to install the full shotgun databases following the "1.2.5 Install databases" in the tutorial. Below is my command and the returns as shown in the terminal:

**vagrant@biobakery:~$ biobakery_workflows_databases --install wmgx
Installing humann utility mapping database
Download URL: http://huttenhower.sph.harvard.edu/humann2_data/full_mapping_v201901.tar.gz
Downloading file of size: 2.55 GB

2.55 GB 100.00 % 9.49 MB/sec 0 min -0 sec
Extracting: /home/vagrant/biobakery_workflows_databases/humann/full_mapping_v201901.tar.gz

Database installed: /home/vagrant/biobakery_workflows_databases/humann/utility_mapping

Unable to write to the HUMAnN config file.
Unable to install database. Error running command: humann_databases --download utility_mapping full /home/vagrant/biobakery_workflows_databases/humann**

May be something is wrong with the HUMAnN? I tried re-downloading the humann but it didn't work. May you please advise how to address this issue?

Thanks in advance

Warm regards

Hong

[ljmciver]

Hi @HongYao4523685 , Thank you for the detailed post and sorry for any confusion. It looks like the database might have been successfully installed at /home/vagrant/biobakery_workflows_databases/humann/utility_mapping but there was an error writing to the config file. With our new vagrant provisioning configuration where we install our packages with pip we use sudo for the install. So since the tool was installed as root an edit to the configuration file will also need to be run as root. If you run the same command again in the VM but run it with "sudo" it should resolve the config write error you are seeing.

Thank you,
Lauren

[hyfood]

Hi Lauren,

Thank you so much!

I tried "sudo biobakery_workflows_databases --install wmgx" and the humann configuration problem has been solved. However, Another error comes. Please see the below for the details:

It seems like it is something to do with the bowtie index...

Thanks again in advance!

Warm regards

Hong

[ljmciver]

Hi Hong, Glad to hear that solved the error! The workflow database install expects the metaphlan database to be installed prior. To install this database run $ metaphlan --install. Once this database is installed it should resolve the errors you are seeing.

Thank you,
Lauren

[hyfood]

Hi Lauren,

Thanks! I run $ sudo metaphlan --install prior to the sudo biobakery_workflows_databases --install wmgx, and the bowtie index problem has been solved. Then several databases were downloaded and extracted, but another error comes:

It seems like it is unable to install the database again.

Could you please advise how to address this issue?

Thanks heaps for your patience
Hong

[hyfood]

Hi Lauren

Finally, the database is successfully installed! The problem is that the biobakery VM image only has the disk size of 64 GB. After I extended it to 256 GB I got no errors!

Thanks for your helps again!!

Hong

[hyfood]

Sorry to disturb again.....

Then, I tried the next step in the tutorial: biobakery_workflows wmgx --input input --output output_data --bypass-strain-profiling, using the samples provided in the tutorial

Some tasks completed, but more tasks failed:

And from the error report, the reason is that the metaphlan_databases directory is not wrieable:

I tried added the sudo prior to the command again but the results are the same.

Could you please suggest how to ideal with this issue?

Warm regards

Hong

[ljmciver]

Hello Hong,

That is great you figured out the disk space issue! The latest version of metaphlan will not check the database installed with a method that requires write permissions. If you update your metaphlan install to the latest version it should resolve the errors you are seeing. Alternatively you can change the permissions on the folder that contains the metaphlan databases.

Thank you,
Lauren

[hyfood]

Hi Lauren,

Thanks for the instructions!

I update the latest metaphlan version and change the permissions as well, and the task 16 was successful. Then all the other tasks run well, but the task 21 which failed:

And the error information is:

Any advise to address this issue?

Thanks again in advance!

Kind regards

Hong

[ljmciver]

Hello @HongYao4523685 , Sorry for my slow response! I have been a bit swamped with a upcoming software deadline. Is it possible that the humann v3 source is no longer installed or that the $PATH or $PYTHONPATH have changed?

Thank you,
Lauren

[hyfood]

Hi Lauren,

Thanks for your response and it is very exciting to hear that your new software is coming!

No need to worry about my issue as I successfully went through the Kneaddata, Metaphlan3 and Humann3 separately in my computer. After coming across some problems, I realize that the problems may come from both the software&database versions and path.

I am grateful for your patience and helps!

Warm regards

Hong