Clean up duplicate code in CI

The ci for this package and the ci in template sources both have a some duplicate code when it comes to tests/builds. These should be cleaned.