Final Submission
How to find my project files and folders
This project was conducted in the qutip-benchmark repository which had been abandoned for 8 years. Rather than updating the existing code I chose to start a new project from scratch, however the pre-existing files and folders are still present in the repository.
Files and folders that existed before the project and were not modified:
Files that existed before the project that I modified:
Files and folders that did not exist and were created during the project:
- qutip_benchmarks contains the python scripts to run and view the benchmarks
- website contains the files used to create the website which hosts the benchmarks
- .github contains the workflows for linting, testing the benchmarks on push/PR and running the nightly benchmarks to be published.
- README.md Description of the python scripts and how to use them
- tutorial.md example use case for pllotting the bencchmarks.
List of PRs created during the project:
- Closed PRs
- Open PRs
Prototype branch
In the first half of the project, before submiting PRs I first created a branch which performed benchmarks and plotted them to test the viability of using git actions for benchmarking, this branch was initially forked and stored in my own github repository here. This branch effectively served as the prototype on which the PRs were based.
Project use cases
1. Running automatic nightly benchmarks in git actions and publishing the results on qutip.org/qutip-benchmark
This job is performed by the nightly_benchmarks workflow. This workflow first downloads the latest version of qutip and the required dependencies into the VM. After this all the previous benchmark result files are downloaded from an S3 bucket, a new benchmark is run using qutip_benchmark.cli.run_benchmarks and the new result file is added to the S3 bucket. The updated benchmark results are then used to produce all the plots with qutip_benchmark.cli.view_benchmarks The plot png files that result from this are placed in the website folder under images/plots. Finally the website folder is published to the gh-pages branch which hosts the qutip-benchmark section of qutip.org
2. Running and plotting benchmarks locally as a user
Information about this can be found in the README.md and tutorial.md files at the root of the repo.
TODO
The project is entirely functional, however some things remain to be done:
- Automatic testing of the plotting and sorting functions in view_utilities.py to facilitate later updates of these functions and uncover potential bugs that have not yet been found.
- Adding more filtering and plotting options to the previously mentionned functions and imporving their user-friendliness.
- Adding more operations and solvers to the benchmarking suite for local users. As the currently available benchmarks are tailored to be performed nightly and published on the website.