I'm ashamed to admit it, but I've never done any automated testing of my code. Usually I'll write a function or method, then do some manual testing and continue on. I realize this is very bad practice so I'm trying to change my ways. Problem is, I'm a bit lost on how to actually start. I'm writing a small Python web application using Flask that is connected to a database and I'd like to use this as a chance to force myself into better development practices.
I'm guessing I need to populate my database with some test data (probably just writing scripts to generate it), then proceed to write tests for each function in my program.
For example: I have a function that pulls all of a users comments from the system. Would I have to write a test for a specific user and then check to make sure the list of returned comment IDs matched an array I created from a manual database query? Is this unit testing?
Just having a hard time understanding it all. Even pointing me towards some newb friendly docs would be very appreciated.
So, just to start with, I would like to mention the purpose of automated testing. The purpose is not to find bugs, but rather to catch regressions. It does have the benefit of catching bugs sometimes, but in general it's a fair amount of work to write and maintain an automated test suite. If you were just out to find bugs, it's far cheaper to just test manually. The advantage of having automated testing is that you can run your suite when you make substantial changes and be more confident you haven't broken the world. This regression testing is the key to why it's worthwhile to write automated tests.
I'm going to describe the methods of testing and then general testing strategies. I'm also just going to explain everything, please don't be offended if you already know half of what I say.
That being said, there are three broad levels of testing:
Unit testing is testing on the smallest scale. It tests individual functions and/or classes. There are many libraries out there that can be used for unit testing, for all sorts of different languages. One example for python is PyUnit.
You generally accomplish unit testing by writing lots and lots of tiny-scoped tests. For example, if you have written your own container class (unlikely in python but it's a simple example), you would have unit tests for adding items, removing items, looking up items, etc.
Component testing is testing your program as individual components. This is generally accomplished by creating mocks of the other components. It's also generally done through the use of a unit testing library, just with larger tests. Where you might have 10's or hundreds of unit tests for each class (largely dependent on the size of the class), you'll have a handful of component tests for each component of your system.
As an example, you mention a database. You could component test by writing a mocked database that returns canned data, and testing the parts that interface with the database using the mock. This is more difficult that unit testing because testability starts to become a concern in the design. Things come up like how are you going to mock a database if the database is hardcoded into your program. It's still relatively easy to do though, it just requires some forethought and planning.
System testing is the hardest to write tests for, because you have to have integration points to drive your system. On a personal project you can probably skip system testing, especially if you've done a thorough job with unit and component testing.
System testing involves going through scenarios on the whole system. The difficulty is vastly increased, especially for a UI application, because you have to have a way to simulate user input. With a text based system, like compilers, you just have to generate text, but with a UI you have to either drive input or have test hooks into the input callbacks. There is no one size fits all solution like there is for unit testing.
Real world testing is completely different than what I thought testing was before I got into the industry. When I was in school I wasted a lot of time doing unproductive things that I thought were productive when I was "testing." That being said, here's my advice:
By a simple google search for 'flask test', the first item returned: http://flask.pocoo.org/docs/testing/
For unit testing, I think you can use some file system based db like SQLite3.
As for how to verify your functions (how to write assert in your test case), this really depends on what's your function is doing. For your specific case, I think you need at least two test cases: one for a user that have a list comments stored, and one for a unknown user, which no comments are expected to return.
you may start by following some online guide about how to write unit tests, like this