Website Documentation

Website Layout

Difficulty Ratings:

Simple: Requires modifying only one file with little-to-no coding
Easy: Some coding involved, minimal effort for basic implementation
Moderate: A good amount of effort and coding will be required, plus rigorous testing and a solid understanding of how the site works (please read this page for help first!)
Hard: This will take a lot of planning, work, testing, and is definitely a project.

Directory Structure

Coming Soon

`assets/scss`

`data/`

`static/`

`static/fonts/`

`static/images/`

`static/css/`

`templates/`

base.html:

`templates/includes/`

`website_utils/`

`app.py` Breakdown

Since our website is a Flask application, app.py is the main program that contains all of the web server functionality: it runs everything!

app.py is broken up into 6 sections, each with different functionality. They are as follows:

Initial Flask Application Setup/Configuration

In order for the site to be recognized as a Flask application, we first will create the standard variable app to store our Flask server instance.
- The __name__ parameter is used to pass the name of the current Python module as the import name to the WSGI application at the center of Flask.

    app = Flask(__name__)

We next call our read_config function which, as detailed in website_utils/config_loader.py, opens up our config.json file stored in the root directory and creates key-value pairs for all json keys and values found within config.json, storing these into the application configuration which can be accessed by referencing app.config inside of app.py.

    read_config(app)

Lastly, we must establish the locations of the assets and static directories.
- The assets directory will be used by the scss_compiler (imported from the flask_scss module) to store our .scss style files, and the static directory will be used by the scss_compiler to generate and compile .css style files from our .scss files and save them into the static/css/ directory.
Then we call the update_scss() method on our scss_compiler in order to actually generate and compile our .scss files into .css files.

    app_dir = os.path.dirname(os.path.abspath(__file__))
    asset_dir = os.path.join(app_dir, "assets")
    static_dir = os.path.join(app_dir, "static")
    scss_compiler = Scss(app, static_dir='static', asset_dir='assets', load_paths=None)
    scss_compiler.update_scss()

Environment-related

We first need to get the location of the seclab log for the timecard, and this gets passed into our memoization function for timecard-related use.

    log_file = MemoizedFile(os.environ.get('BADGE_LOG_PATH', 'seclab.log'))

Next, some special stuff occurs (see Webmaster documentation).

Site Pages

This section contains all of the routing for the different resources our website will serve through Flask.
First, we establish the routing for serving the index page ('/') and all other valid pages (as listed in the data/pages.json file), or 404 errors otherwise.
- Every page request first makes a request to the API correlated API endpoint (therefore the page's url and the API endpoint's url must be the same or expected functionality/content won't be present). This retrieves the content needed to then render the desired page's template with, as passed in via the page parameter page_data.
We then route all of the resource-related file extensions that may be needed to be accessible by our site.
- This includes: .txt (humans.txt and similar files), .jpg (some photos), .png (more photos), .ico (the browser favicon), .css (stylesheets), .otf (font files), and .js (javascript).
We also setup routing needed for the Timecard and offline page loading (partial implementation)
Also, some other stuff is documented in the Web Master Documentation file.

API Endpoints

In order to maintain compatibility across versions, we have a version number specified first.
- With major API upgrades, this version number should be incremented, while also keeping the prior version live as well.

    version = 1

We then take this version number and build our API endpoint URL prefix:

    apiurl = '/api/' + 'v' + str(version)

Next, there are two Flask routes present for the API.
The first is the standard /api route which returns a welcome message to the client and no data
The second is the main functionality behind all other API endpoints. This route is flexible to iterate through the data present in data/api.json and determine what data should be returned to the client, if the API endpoint is valid, and if any other functionality should be done as well.
- see Adding a New API Endpoint for more information

Lastly, there are endpoint-related functions which have been created for their respective API endpoints to call.
- These may contain a single parameter, filename, which specifies the file that should be opened and used by the API endpoint function

Error Routing

Currently, we are only handling all 404 errors and returning the 404.html template view, however more error routing functionality can be extended and implemented here.

Utility Functions

Here, functions which are necessary for backend-related functionality are present.
Currently, this includes a token-refreshing function (to handle expiration of our authentication token for Google Calendar event access), a check_calendar function which currently checks if there is an event on the current day, and the video_writer function which ensures that our server-side storage (data/videos.json) is always up-to-date with newly-published videos from our YouTube channel.
There also is the @app.context_processor declaration and the utility_processor function which, currently, is only used to ensure that any template page has access to checking if there is an event on the current day (allowing for the "notification banner" to be present on all pages if so desired in the future).

And, of course, the last lines:

    if __name__ == '__main__':
        app.run(debug=True, host='0.0.0.0', port=3000)

which runs the application (if run from the commandline by python3 app.py within the virtualenv) in debug mode, broadcasting the website (0.0.0.0), and on port 3000.

Site Styling

The site's style sheets are located in assets/scss. SCSS is an extension of CSS that allows you to write CSS stylesheets with less hassle. The SCSS style sheets get compiled to regular CSS when running app.py and are placed in static/css. If you'd like to tinker with the style sheets, delete the current CSS files with rm static/css/* before running app.py to make sure the changes are refreshed.

The Website

Adding a New Page (Option 1)

Difficulty: Moderate

In order to add a new page to the website, the following steps must be followed in order for desired functionality to be present and for coherence with the rest of the published site.

Design the page itself with the following in mind:
- What is the purpose of the page? What content will be hosted on this page?
- Does this page have any overlap in other pages? (Can the new content be combined into a previously-created page?)
- What should the page's URL be, and is it related to the page's purpose?
- Does it need a custom style or is it purely containing text as content? (if it needs a custom style then skip to the next section)

Create the page's .json file and save it in the data/ directory.
- This can be in any format desired, as long as it is valid JSON.
- The naming convention for the file should be, (substituting text found in <>): <page-name>.json
Create an API endpoint for the page to handle its content (see Adding a New API Endpoint).

Create a .scss file for the page and save it in the assets/scss/ directory.
- This file, without any styling, should be as follows:
```
    @import 'style';
```
- The naming convention for the file should be: <page-name>.scss

Add a else-if statement to the templates/includes/header.html file in order for the page to use the .scss file created in the above step.

The format should be as follows: (substituting text found in <>)

    ...
    <!-- per-page css -->
    ...
    {%- elif request.path == "/<page-url>" %}
    <link rel="stylesheet" type="text/css" href="/<page-name>.css" />
    ...
    {%- else %}
    <link rel="stylesheet" type="text/css" href="/style.css" />
    {% endif %}
    ...

Add the desired url for the page to the list found in the data/pages.json file.

This should conform to the following format: (substituting text found in <>)

    {
        "pages": [
            ...
            "<page-url>",
            ...
        ]
    }

Create the .html file for the page in the templates/ directory.
- The name of the file should be the name of the url, (substituting text found in <>): <page-url>.html
- The file contents must contain, at the least, the following: (substituting the <page-content> text with the page's contents)
```
    {% extends "base.html" %}
    {% block body %}
        <div class="content">
            <div class="content" id="landing-bg"></div>
            <div id="welcome-bar">
                <h1 id="main"><page-content></h1>
            </div>
        </div>
    {% endblock %}
```
- Any page content that you wish to display should be passed in from the API endpoint's results. A variable page_data is made available to the page template .html files, and this variable contains the exact data present as the value of the "data" key in the API endpoint's results. (see templates such as templates/videos.html for example usage)
- This page can also be further customized as necessary, as long as all html code is within the {% block body %} and {% endblock %} area.

Adding a New Page with Custom Styling (Option 2)

Difficulty: Moderate/Hard

Coming Soon

Updating Any Page's Contents

Difficulty: Simple

Determine the location of the contents for the page that needs updating.

The contents should be stored in a .json file within the data/ directory.

Update/modify the contents accordingly.

NOTE: some data .json files have specific purposes and should not be modified without first consulting the rest of this document to determine if there are different instructions needed

Commit changes.
git push origin master && git push production master
Ensure the changes are now live on that page of https://thewhitehat.club

Refreshing the Officers Page

Difficulty: Simple/Easy

Update the information for the positions in data/officers.json

The format for each officer position should be as follows: (substituting text found in <>)

    ...
    "<position-name-short>": {
        "position": "<official position title>",
        "name": "<officer's name>",
        "image": "<image_name>.jpg"
    },
    ...

see data/officers.json for live examples

Replace the images stored in static/images/officers/ with images of the new officers.

The webserver (app.py) will automatically prepend officers- in front of the images when the /officers page is loaded. This is done in order to differentiate between images which are the officer photos in static/images/officers/ and images which belong to the rest of the site and should be loaded from static/images/.
- This functionality is found within templates/officers.html.
NOTE: The name of the image file MUST be the same as the value of the image key within the data/officers.json file, otherwise things will not work as intended!
- For convention, the image file names should be (in lowercase) the name of the person for each position (ex: jon.jpg)

Commit changes.
git push origin master && git push production master
Ensure the refreshed changes are now live on /officers.

Updating the Timecard

Difficulty: Simple

Add a new entry to data/timecard_dates.json for the current quarter.

The format should be as follows: (substituting text found in <>)

    ...
    {
        "quarter": "<quarter name> <year>",
        "start": "<start date in yyyy-mm-dd format>",
        "end": "<end date in yyyy-mm-dd format>"
    }
}

see data/timecard_dates.json for live examples

Remove the entry for the oldest quarter from data/timecard_dates.json
Commit changes.
git push origin master && git push production master
Ensure the updates are now live on /timecard.

TODO:

Add a way to view custom time durations on the timecard (ex. select prior quarters from a dropdown menu). (suggestion)

Adding New Resources to the Resources Page

Difficulty: Simple/Easy

Collect information about the new resource including:

which category it falls into: Tools, Links, Books, or other (current as of writing this page).
what the displayed name of the resource should be
a safe and secure link to the resource online (we don't want to be spreading around unsafe links)
a friendly description about what the resource is/does, and why people in the club should use it

Add the resource and its information to the data/resources.json file in the following format: (substituting text found in <>)
- NOTE: the current implementation of the website allows for HTML code to be embedded only within the "description", allowing for clickable links to be present within the description if desired.
```
    ...
    {
        "name": "<displayed-resource-name>",
        "link": "<resource-url>",
        "description": "<a-friendly-description!>"
    },
    ...
```
- see data/resources.json for live examples
Commit changes.
git push origin master && git push production master
Ensure that the new resource is now live on /resources.

The API

Adding a New API Endpoint

Difficulty: Simple/Easy

Determine whether the new API endpoint will (a) only return content visible on the rendered website, or (b) expose new functionality to users.

if (a):

Add the API Endpoint to data/api.json in the following format: (substituting text found in <>)

NOTE: the content file must exist within the data/ directory and must be a valid .json file!

    ...
    "<api-url>": {
        "message": "<an-informative-message-to-the-user>",
        "file": "<.json-datafile-to-be-returned>"
    },
    ...

see data/api.json for live examples (ex. the "visit" API endpoint)

else if (b):

To make the API Endpoint perform additional functionality, more is required.

2a. Create the function to perform the desired functionality within app.py (preferably organized with the other API functions). 2b. Add the API Endpoint to data/api.json in the following format: (substituting text found in <>)
- NOTE: the value for the "function" key must be the name of the function just created within app.py, including the parentheses ().
```
    ...
    "<api-url>": {
        "message": "<an-informative-message-to-the-user>",
        "function": "<function-name-in-app.py>()"
    },
    ...
```
- see data/api.json for live examples (ex. the "ls" API endpoint)
- This functionality (executing the function detailed in data/api.json) comes by executing the eval() function in Python on the string stored within data/api.json as its parameter. While this would be a bad security practice in most cases (executing an arbitrary function given some user input), the usage here is relatively secure, since user input is only used to find the necessary key within data/api.json in order to then pull data from. No user input ever gets passed into eval(), only the value of the function key as defined in data/api.json.
  - This requires that the Webmaster thoroughly vets and reviews all functions referenced from data/api.json and the related code being executed within app.py so no webserver errors or security bad practices occur (as their position already entails).
2b. (optional) if the new function needs information from a file within data, then the json for the API endpoint should look as follows: (substituting text found in <>)
- NOTE: the content file must exist within the data/ directory and must be a valid .json file!
```
    ...
    "<api-url>": {
        "message": "<an-informative-message-to-the-user>",
        "function": "<function-name-in-app.py>()",
        "file": "<.json-datafile-to-be-passed-to-function>"
    },
    ...
```
- see data/api.json for live examples (ex. the "videos" API endpoint)
- The functionality that allows for other files to be passed as arguments in to the new function comes from some string manipulation, present in the api function in app.py, before the call to eval().

endif

Rigorously test the API endpoint locally (of course)
Commit changes.
git push origin master && git push production master
Ensure new endpoint is now live.

Updating/Modifying an API Endpoint

Difficulty: Easy/Moderate

Find the API Endpoint that needs changes within data/api.json.

Make the changes necessary to the API, ensuring the end result is of the format: (substituting text found in <>)

    ...
    "<api-url>": {
        "message": "<an-informative-message-to-the-user>",
        "function": "<function-name-in-app.py>()",
        "file": "<.json-datafile-to-be-passed-to-function>"
    },
    ...

see data/api.json for live examples

Test the API endpoint locally (of course)
Commit changes.
git push origin master && git push production master
Ensure desired changes are now live.

Major API Changes

Difficulty: Moderate/Hard

The only thing necessary to be done before implementing new major API changes is to:

Increment the version number in app.py
Ensure that the previous API still is accessible at the old version number URL, for initial backwards-compatibility support.
Begin migration process of all services dependent on the old API's services.
Development.
Update the documentation to reflect any changes (as necessary).
Commit changes.
git push origin master && git push production master

Adding content to the CTF

Difficulty: ???

What CTF? Stop snoopin'...

Consult the WebMaster documentation

Website Documentation

Website Layout

Difficulty Ratings:

Directory Structure

assets/scss

data/

static/

static/fonts/

static/images/

static/css/

templates/

templates/includes/

website_utils/

app.py Breakdown

Initial Flask Application Setup/Configuration

Environment-related

Site Pages

API Endpoints

Error Routing

Utility Functions

Site Styling

The Website

Adding a New Page (Option 1)

Difficulty: Moderate

Adding a New Page with Custom Styling (Option 2)

Difficulty: Moderate/Hard

Updating Any Page's Contents

Difficulty: Simple

Refreshing the Officers Page

Difficulty: Simple/Easy

Updating the Timecard

Difficulty: Simple

Adding New Resources to the Resources Page

Difficulty: Simple/Easy

The API

Adding a New API Endpoint

Difficulty: Simple/Easy

Updating/Modifying an API Endpoint

Difficulty: Easy/Moderate

Major API Changes

Difficulty: Moderate/Hard

Adding content to the CTF

Difficulty: ???

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

`assets/scss`

`data/`

`static/`

`static/fonts/`

`static/images/`

`static/css/`

`templates/`

`templates/includes/`

`website_utils/`

`app.py` Breakdown