Web application: end-to-end system setup

Goals

• Assemble the skeleton of your web application so the user can load a page and see some data returned by your API.
• Think about how a rudimentary but fully functional end-to-end app supports iterative development.

Rubric

Where are we?

So far, you have:

• Picked a dataset
• Thought about the ways in which a person might want to interact with your data
• Sketched wireframes of your user interface
• Learned how to implement an API in Flask
• Taken a look at a sample web app that combines HTML pages and API queries in a single Flask app

To complete your project, you need to assemble all these pieces plus some HTML/CSS/Javascript into a functional system, and then refine it to do what you have envisioned (and to maybe do it better than you envisioned, since you'll have new ideas as you progress).

In my experience, this kind of project goes best if you use a light-weight iterative development process roughly like this. First, you set up a minimal version of your application that includes all the architectural components (e.g. a front-end page, a back-end server, a database, and an API) and the simplest possible communications between them (e.g. a little Javascript that calls a "hello"-style API endpoint, etc.). This baseline version of your application has essentially no features, but it has the architecture you need, and you can run it. Once you have this runnable shell of your program, you start "iterating":

1. Identify a small addition to your project that will bring your application a little bit closer to its target. (Try to identify tasks that will take between 1 minute and 2 hours to do. Breaking your project down into small steps like this is sometimes hard, but it's usually possible, and always powerful.)
2. Implement it
3. Test the updated project and debug as necessary.
4. Commit, pull, push.
5. Done? Great! Otherwise, go to step 1

This current assignment's goal is to get you into this development loop.

(Of course, it's not quite that simple. Sometime during the iterations you need to do usability testing, code reviews, alpha testing, beta testing, etc. But the basic cycle of "small goal, small change, test, repeat" is very effective overall.)

How I'll test your submissions

To test this assignment, I will:

• cd YOUR_REPO
• git pull
• Load your data into my database: psql -U jondich grading < data/database.sql
• cd webapp
• Edit config.py to work with my postgresql setup:
  user='jondich' password='' database='grading'
• Run your webapp: python3 app.py 127.0.0.1 5000
• View your website in Chrome: http://127.0.0.1:5000/ (note that I expect your site's home page to be at the / route)
• Click on the "Get data" button to see how your Javascript interacts with your API
• Take a quick spin through your code to make sure the pieces are as I expect them to be: HTML, CSS, Javascript, Python, and SQL.
• Restore my postgres setup by deleting your data: DROP TABLE your, tables, go, here;

The reason I'm describing this testing procedure is so that you can, if you wish, test this procedure yourself to make sure it works.

Your tasks

• If you haven't done so yet, do watch this video about our web app architecture. NOTE: This video is a few years old, but is basically still good for our purposes. The web app that I use in the video is now located in my repo in webapps/app4-for-video.
• Set up your web app's structure to mirror the structure of webapps/app3-api-and-db, which is simpler and a bit better than app4-for-video. In particular, name your web app's files and directories, inside the webapp directory, like so:
  • app.py: the Flask app containing routes that serve HTML pages
  • api.py: the Flask blueprint containing routes that serve the API endpoints
  • static/: the directory that contains your .css and .js files, and any other static files (e.g. images) that you use
  • templates/: the directory that contains your .html files (which may or may not contain Flask/Jinja2 template code
  • templates/index.html: your home page (template or not)

  Note that I expect you to use exactly these names (e.g. app.py, api.py, etc.) and locations (do put this stuff directly in webapp/, not in a subdirectory of webapp/). This is a big deal for automating my processes for giving you timely feedback on various stages of your project between now and the end of the term. I'll be grateful for your help on this.

• Dump your database into the file data/database.sql:

  pg_dump --no-owner --no-privileges --clean -U YOURUSERNAME YOURDATABASE > data/database.sql
  Doing this gives me a quick way to load your data into my own postgres database. Here, YOURUSERNAME and YOURDATABASE are the values you use when you launch psql. If your setup doesn't require you to specify your user name for psql, then skip it for pg_dump as well.

  NOTE: If your database.sql file is larger than 25MB, then don't add it to your repository. Instead, DM it to me via Slack.

• Make sure that python3 app.py 127.0.0.1 5000 works.

• When you're ready and you have add/commit/pull/push'ed all your files to your repository, tag the repository:

  git tag -a end2end -m "the end-to-end assignment" git push origin end-to-end

  Reminder: that special git push is because the normal "git push" doesn't push tags.

What do I expect again?

• Your web app runs (python3 app.py 127.0.0.1 5000)
• It gives me a web page at http://127.0.0.1:5000/
• It gives me a "Get data" button to allow me to see data that came from your database via a Javascript query to your API, which in turn accesses the database via psycopg2.
• Any such data will be sufficient. Just a a simple list of things (national parks or countries or songs or...) is absolutely sufficient for this assignment.
• The only thing I care about for this assignment is that you have this end-to-end structure involving the Javascript, the API, and the database up and running so you can start building on top of that structure. What your app actually does with that end-to-end structure is immaterial to me for this assignment.

How would I pursue this if I were you?

• I would start with the files from webapps/app3-api-and-db. Just copy those babies into your own webapp/ folder.
• Then edit api.py to delete my sample routes and replace them with your already-implemented API routes from the previous homework.
• Move static/books.js and static/books.css to files with better names for your project. Just webapp.js and webapp.css would be fine. In the steps below, you'll want to catch any references to the books files and change them to the new names.
• Next, I would replace the body of templates/index.html with a rudimentary home page based on your wireframes (or whatever your vision has evolved to since you created the wireframes). And similarly with any CSS you develop—put your CSS into the static/ folder (or even just paste your CSS into webapp.css file, replacing my CSS). Similarly with any Javascript you might have written.
• Add your easiest-to-implement API route to api.py.
• Edit index.html and static/webapp.js to give the user a way to see some data from your database.
• Test it, do the pg_dump, add/commit/pull/push, tag + tag-push, call it good and move on.

Keep those questions coming, and have fun!