Act now before dependencies lead workflows astray

Breeze can give you a list of all the files, network connections, programs and libraries that an application or workflow uses. This is a very powerful tool for preventing problems, but all too often it’s only used when a program goes wrong or when migrating an application to a new compute environment or to the cloud.

When applications read the wrong files or store data in the wrong place this can cause everyday performance issues, as well as lead to legacy portability problems.

For example, these are a few problems uncovered by our customers:

  • tmp files being stored on the networked file system instead of on local disk
  • The wrong extension being used in a genome pipeline that was invalidating the organisation’s meta-data and backup management
  • A workflow with a hard coded path to a home dir, which meant it only ran in the US
  • A workflow in which a library dependency had been removed but where it actually still accessed it in some places.

Solution: Carry out a dependency audit

Some dependency checks can be automated and integrated into your quality assurance flows and regression tests. Other checks need a human who understands the application to look at the dependencies and check that they match their view of what the application should be doing. The best solutions therefore use a mix of these approaches.

1. Introduce dependency checks into your quality assurance (QA) tests. It’s simple to draw up forbidden areas of the file system and run a cross check of this against the dependencies as produced with Breeze.

Breeze can also check for other problems such as executables being sourced from different places or when applications mix versions of a particular program. The Healthcheck report, available with Breeze, gives you a good head start on what to look for.

2. Introduce user or developer-driven audits on a regular basis. No amount of automated testing can completely remove the need for a human to look at a program and check that it’s behaving as it should. Automated tests can completely miss issues such as a GUI suddenly turning purple, for example!

We recommend running a manual dependency audit with at least one real-life scenario of your application before running any major release tests. This gives you the opportunity to fix anything before release and gives the best chance of applications passing the tests first time.

Finding answers

It is easy to find the list of dependencies in the Healthcheck report. Simply run the trace from Breeze through Healthcheck and scroll down to the section on Home Directories and Temporary Files.

If you need more information, you can find more information on specific files and programs accessed in Breeze, using the File view.