Sample Programs 25 Project Tests Release

Once again, we’re back with yet another Sample Programs update! If you were following along during Hacktoberfest 2020, then you might know that we hit one of our latest milestones: 25 project tests. Don’t worry if you don’t know what that means! We’ll talk all about it.

Table of Contents

What Are Project Tests?

For the uninitiated, project tests are basically Python unit tests written with the assumption that every project will handle input and output on the command line. Currently, this is all managed through Glotter, a testing framework using Docker.

While future iterations of Glotter may eliminate the dependency on Python altogether, we’re stuck writing our own tests for the time being. A typical test file looks as follows:

from runner import ProjectType
from glotter import project_test, project_fixture


@project_fixture(ProjectType.HelloWorld.key)
def hello_world(request):
    request.param.build()
    yield request.param
    request.param.cleanup()


@project_test(ProjectType.HelloWorld.key)
def test_hello_world(hello_world):
    actual = hello_world.run()
    assert actual.strip() == 'Hello, World!'

Then, when the project is being built, the test will be executed for all languages that include a Glotter configuration file. For example, Python’s Glotter config file looks like this:

folder:
  extension: ".py"
  naming: "underscore"

container:
  image: "python"
  tag: "3.7-alpine"
  cmd: "python {{ source.name }}{{ source.extension }}"

In the Sample Programs repo, we actually launch Glotter during our Travis builds which look like this:

$ python runner.py test
============================= test session starts ==============================
platform linux -- Python 3.7.1, pytest-5.2.4, py-1.8.0, pluggy-0.12.0 -- /home/travis/virtualenv/python3.7.1/bin/python
cachedir: .pytest_cache
rootdir: /home/travis/build/TheRenegadeCoder/sample-programs
collecting ... collected 1853 items
test/projects/test_baklava.py::test_baklava[baklava.go] PASSED           [  0%]
test/projects/test_baklava.py::test_baklava[Baklava.kt] PASSED           [  0%]
test/projects/test_baklava.py::test_baklava[Baklava.fs] PASSED           [  0%]
test/projects/test_baklava.py::test_baklava[baklava.f95] PASSED          [  0%]
test/projects/test_baklava.py::test_baklava[baklava.swift] PASSED        [  0%]
test/projects/test_baklava.py::test_baklava[baklava.ex] PASSED           [  0%]
...
test/projects/sorting/test_sleep_sort.py::test_sleep_sort_valid[sleep-sort.dart-sample input: already sorted] PASSED [ 99%]
test/projects/sorting/test_sleep_sort.py::test_sleep_sort_valid[sleep-sort.dart-sample input: reverse sorted] PASSED [ 99%]
test/projects/sorting/test_sleep_sort.py::test_sleep_sort_invalid[sleep-sort.dart-no input] PASSED [ 99%]
test/projects/sorting/test_sleep_sort.py::test_sleep_sort_invalid[sleep-sort.dart-empty input] PASSED [ 99%]
test/projects/sorting/test_sleep_sort.py::test_sleep_sort_invalid[sleep-sort.dart-invalid input: not a list] PASSED [ 99%]
test/projects/sorting/test_sleep_sort.py::test_sleep_sort_invalid[sleep-sort.dart-invalid input: wrong format] PASSED [100%]
======================= 1853 passed in 832.41s (0:13:52) =======================
The command "python runner.py test" exited with 0.

Naturally, putting together tests for 25 different projects is quite the feat, so we wanted to celebrate it with a little milestone.

Project Testing Challenges

When it comes to writing tests for each project, there are few challenges.

First, before we can begin to write tests, we must write documentation. During the documentation process, our authors are expected to also write up a testing table which is ultimately converted into testing. This process is long and iterative as we need to be very careful that testing is both complete and correct—it’s not easy to make changes once testing is official.

Second, we must write the tests. For the most part, testing is a matter of defining inputs and outputs, but creating the files is a bit of a process. In particular, there are three files that need to be updated:

• .glotter.yml
• test_project.py
• runner.py

Assuming everything is done correctly, the next challenge is making sure all existing scripts adhere to testing. Ideally, we would implement testing before we let folks submit code, but we’ve largely been adding tests retroactively. As a result, a lot of code snippets usually need to be updated. This is a long and challenging process; it’s why it’s taken over a year just to cover 25 projects.

Finally, the biggest challenge has been enforcing our requirements. Generally, folks like adding new projects to their favorite languages, and the more popular the language, the more often this happens. However, we don’t want to have folks creating unmanageable technical debt for us, so we ask that they both document and test their code. Unfortunately, this turns a lot of folks away, so it’s been tough.

That said, I’m just happy to have hit our milestone of 25 project tests. Speaking of that, let’s talk about it!

25 Project Tests

As the title of this article suggests, we finally hit our project testing milestone during Hacktoberfest 2020. Here’s the official list of projects that are now tested:

• Baklava
• BinarySearch
• BubbleSort
• Capitalize
• ConvexHull
• EvenOdd
• Factorial
• Fibonacci
• FileIO
• FizzBuzz
• HelloWorld
• InsertionSort
• JobSequencing
• LCS
• LinearSearch
• MergeSort
• MST
• Prime
• QuickSort
• Quine
• ROT13
• ReverseString
• RomanNumeral
• SelectionSort
• SleepSort

For a complete and current list of projects, check out the official projects page.

Next Stop: 50 Project Tests

As you know, the grind never stops! Now that we’ve covered 25 of our projects, what’s stopping us from hitting 50? These days I’m a bit too busy to add the tests myself, but I’m totally open to support from the community. Anyone want to help out?

Conveniently, there are more ways to help out than submitting code. You can always throw the repo a star. Or even better, check out my list of ways to grow the site. These days you can find my Discord, Patreon, and Newsletter there.

Otherwise, you can read all about other Sample Programs updates below:

• The Sample Programs Website Automatically Generates Featured Images
• The Sample Programs Repo Squashes Commits

As always, thanks for stopping by! I appreciate it.

Sample Programs Repo News (19 Articles)—Series Navigation

Everyone once in awhile, I like to update y’all on what’s going on in the Sample Programs repo. At this point, I’ve written quite a few updates, so I figured it might be helpful to group them as a series.

← Previous Post: [#12] [#14]: Next Post →