diff --git a/ExpressAPI/assets/OpenAPI/OpenAPI.yaml b/ExpressAPI/assets/OpenAPI/OpenAPI.yaml index f42ac3919c80960bfd145fdfb69c8da9ed6be28b..2bbe1d3a26375c79d0b02d84bb0773b74d13712c 100644 --- a/ExpressAPI/assets/OpenAPI/OpenAPI.yaml +++ b/ExpressAPI/assets/OpenAPI/OpenAPI.yaml @@ -1,7 +1,7 @@ openapi: 3.1.0 info: title: Dojo API - version: 3.4.0 + version: 3.4.2 description: | **Backend API of the Dojo project.** diff --git a/Wiki/Development/1-How-to-setup-your-development-environment.md b/Wiki/Development/1-How-to-setup-your-development-environment.md new file mode 100644 index 0000000000000000000000000000000000000000..170be18b960af73c24a268ce8363cd5d0162ed33 --- /dev/null +++ b/Wiki/Development/1-How-to-setup-your-development-environment.md @@ -0,0 +1,131 @@ +# How to setup your development environment + +## Introduction + +This tutorial describes how to setup your development environment for building the Dojo API by detailing the +prerequisites and dependencies needed. + +## Technologies + +The API is built using [NodeJS](https://nodejs.org/en/) and [NPM](https://www.npmjs.com/). + +The programming language used is [Typescript](https://www.typescriptlang.org/) v5. + + +## Prerequisites + +In order to build the API you will need the following tools: +- [NodeJS](https://nodejs.org/en/) (version 20 or higher) +- [NPM](https://www.npmjs.com/) (version 10 or higher) + +Install NodeJS and NPM by following the instructions on the [official website](https://nodejs.org/en/download/package-manager). +Or via Node Version Manager (NVM) by following the instructions on the [official website](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm). + + +## Dependencies + +Here are the main dependencies used by the API (you do not need to install them manually or globally on your system): +- [Axios](https://www.npmjs.com/package/axios): a promise-based HTTP client for the browser and Node.js. It is +used to make HTTP(S) requests to the Dojo backend and Gitlab. +- [Dotenv](https://www.npmjs.com/package/dotenv): used to load environment variables from a .env file. +- [Dotenv-vault](https://www.npmjs.com/package/dotenv-vault): a CLI to sync .env files across machines, +environments, and team members. +- [Express](https://www.npmjs.com/package/express): a minimal and flexible Node.js web application framework that +provides a robust set of features for web and mobile applications. Used to create the API server. +- [express-validator](https://www.npmjs.com/package/express-validator): used to validate and sanitize express requests. +- [Morgan](https://www.npmjs.com/package/morgan): used to log HTTP requests. +- [JsonWebToken](https://www.npmjs.com/package/jsonwebtoken): used to generate and validate [JSON Web Tokens](https://jwt.io/). +- [Prisma](https://www.npmjs.com/package/prisma): a modern database access & ORM for Node.js. Used to construct (from +scratch and migrations) and interact with the database. +- [Winston](https://www.npmjs.com/package/winston): used for logging purposes. Used in combination with Morgan to +log HTTP requests in the format defined with Winston. +- [zod](https://www.npmjs.com/package/zod): a TypeScript-first schema validation with static type inference. Used +in the projet to validate json files created by the user. + + +## Installation + +First of all, you need to clone the repository: +```bash +$ git clone --recurse-submodule https://gitedu.hesge.ch/dojo_project/projects/backend/dojobackendapi.git +``` + +Then, you need to move to the project's directory: +```bash +$ cd dojobackendapi/ExpressAPI +``` + +To install the dependencies listed above you can use the following command in the base directory of the project: +```bash +$ npm install +``` + + +## Environment variables + +Environment variables are used to store sensitive information such as API keys, passwords, etc. +They are also used to store configuration information that can be changed without modifying the code. + + +### Using on your development machine + +To use environment variables on your development machine, you need the `.env.keys` file in addition of the `.env.vault` +file present in the repository. + +**The `.env.keys` file have to be requested to the project maintainer: [Michaël Minelli](mailto:dojo@minelli.me).** + + +### Decrypting the environment variables + +You can decrypt env var stored in the `.env.vault` file with the following commands in the project's main folder. +Here is an example of how to decrypt the environment variables for the development environment: +```bash +$ npx dotenv-vault local keys + environment DOTENV_KEY + ─────────── ───────────────────────────────────────────────────────────────────────── + development dotenv://:key_1234@dotenv.local/vault/.env.vault?environment=development + +Set DOTENV_KEY on your server + +$ npx dotenv-vault local decrypt 'dotenv://:key_1234@dotenv.local/vault/.env.vault?environment=development' > .env.development +``` + + +### How to modify the environment variables + +You can modify environment variables in `.env` files that you have previously decrypted. +You also need to re-encrypt the modified `.env` files to store them in the `.env.vault` with the following command: +```bash +$ npm run dotenv:build +``` + + +## Database + +For the development, you can use the docker compose file in the `Resources/DevInfra/` folder. + +The following command have to be executed in the base directory of the repository: +```bash +$ docker compose -f Resources/DevInfra/docker-compose.yml up -d +``` + +This will run a MariaDB database on port `59231` with the following credentials: `root:9buz7f312479g6234f1gnioubervw79b8z` + +A second container is created with the Adminer tool on port `62394`. + +#### Structure creation and seeding + +The following command (to be executed from the `ExpressAPI` folder) will create the database structure and seed it with +some example data. + +```bash +$ npm run database:deploy:dev +``` + + +## Run the API + +To run the API (in dev mode) you can use the following command in the base directory of the project: +```bash +$ npm run start:dev +``` diff --git a/Wiki/Development/2-How-to-add-a-new-route.md b/Wiki/Development/2-How-to-add-a-new-route.md new file mode 100644 index 0000000000000000000000000000000000000000..08df03d8e822dd37fe0a5cd5e1c8bc175eb0772a --- /dev/null +++ b/Wiki/Development/2-How-to-add-a-new-route.md @@ -0,0 +1,410 @@ +# How to add a new route to the Dojo API + +## Introduction + +This tutorial describes how to add a new route to the Dojo API. For that we take two existing routes and describe +his implementation. This route allow a member of the teaching staff of an assignment to publish / unpublish it. + + +## Prerequisites + +All the prerequisites are described in +[How to setup your development environment](1-How-to-setup-your-development-environment) tutorial. + + +## Properties of the new route + +### Description of the route +- `Route :` /assignments/:assignmentNameOrUrl/publish +- `Verb :` PATCH +- `Resume :` Publish an assignment +- `Protection type :` Clients_Token +- `Protection :` TeachingStaff of the assignment or Admin role + +### Params of the request (url params) +- `Name :` assignmentNameOrUrl + - `Description :` The name or the url of an assignment. + - `Location :` Query + - `Required :` Yes + - `Data type :` string (path) + +### Possible Response(s) +- `Code :` 200 + - `Description :` OK + - `Content of the response :` +```json +{ + "timestamp": "1992-09-30T19:00:00.000Z", + "code": 200, + "description": "OK", + "sessionToken": "JWT token (for content, see schema named 'SessionTokenJWT')", + "data": {} +} +``` +- `Code :` 401 - Standard +- `Code :` 404 - Standard + + +## Routes files structure + +The routes files are located in the `src/routes` folder. All routes files are named with the following pattern: +`SubjectRoutes.ts` where `Subject` has to be replaced by the general subject of the routes implemented in it (f.e. +Exercise, Assignment, Session, etc.). + +### Application to our use case +In our case we will add our route to the file with the following path : +`src/routes/AssignmentRoutes.ts`. + + +## Routes class inheritance + +All routes files must inherit from the `RoutesManager` interface. This interface is located in the +`src/express/RoutesManager.ts` file. + +When you inherit from this interface you will need to implement the following method : +- `registerOnBackend(backend: Express): void;`: This method get the express backend object for register new routes. + +### Apply to our use case + +Now, the `src/routes/AssignmentRoutes.ts` file will look like this: +```typescript +import RoutesManager from '../express/RoutesManager'; + +class AssignmentRoutes implements RoutesManager { + protected commandName: string = 'publish'; + + registerOnBackend(backend: Express) { + ... + backend.patch('/assignments/:assignmentNameOrUrl/publish', ..., this.publishAssignment.bind(this)); + backend.patch('/assignments/:assignmentNameOrUrl/unpublish', ..., this.unpublishAssignment.bind(this)); + } + + protected async commandAction(assignmentNameOrUrl: string, options: { force: boolean }): Promise<void> { + private async publishAssignment(req: express.Request, res: express.Response) { + return this.changeAssignmentPublishedStatus(true)(req, res); + } + + private async unpublishAssignment(req: express.Request, res: express.Response) { + return this.changeAssignmentPublishedStatus(false)(req, res); + } + + private changeAssignmentPublishedStatus(publish: boolean): (req: express.Request, res: express.Response) => Promise<void> { + return async (req: express.Request, res: express.Response): Promise<void> => { + ... + }; + } + } + +export default new AssignmentRoutes(); +``` + + +## Define request param binding + +The goal of the step is to define a generic request parameter and bind functions that will complete a property in the +`request.boundParams` object. +For exemple, we can define a function that will take the `assignmentNameOrUrl` parameter and find the assignment +corresponding to it, add it to the request object and call the next function. + +Steps : +1. First of all, we need to add the new property to the type located in the `src/types/express/index.d.ts` file in the +boundParams object with type: `Type | undefined` with `Type` is the type of the param. + +2. In the `src/middlewares/ParamsClallbackManager.ts` file we need to: + 1. Add the new property to the `boundParams` object in the `initBoundParams` method with the `undefined` value. + 2. Call the the `listenParam` method in the `registerOnBackend` method for each paramer as follows: + ```typescript + this.listenParam(paramName: string, backend: Express, getFunction: GetFunction, arrayOfArgsForGetFunction: Array<unknown>, boundParamsIndexName: string + +### Application to our use case + +1. In the `src/types/express/index.d.ts` file: +```typescript +declare global { + namespace Express { + interface Request { + ... + boundParams: { + ... + assignment: Assignment | undefined + }; + } + } +} +``` + +2. In the `src/middlewares/ParamsClallbackManager.ts` file: +```typescript +... +initBoundParams(req: express.Request) { + if ( !req.boundParams ) { + req.boundParams = { + ..., + assignment: undefined + }; + } +} + +registerOnBackend(backend: Express) { + ... + + this.listenParam('assignmentNameOrUrl', backend, (AssignmentManager.get as GetFunction).bind(AssignmentManager), [ { + exercises: true, + staff : true + } ], 'assignment'); +} +``` + + +## Add security to the route + +A middleware is available to check permissions. It is located in the `src/middlewares/SecurityMiddleware.ts` file. +This file does not need to be edited unless you want to add some new security tests. + +You can use the function `SecurityMiddleware.check(checkIfConnected: boolean, ...checkTypes: Array<SecurityCheckType>)` +function as a middleware in the route definition. + +You have the possibility to just check if the user is connected with the first parameter. If you want to add a more +specific check you can add some parameters with the `SecurityCheckType` enum value (f.e. teaching staff, assignment +staff, etc.). + +**WARNING:** The `SecurityCheckType` args array is interpreted as an `OR` condition. So if you call : +`SecurityMiddleware.check(true, SecurityCheckType.TEACHING_STAFF, SecurityCheckType.ASSIGNMENT_STAFF)`, the middleware +will check if the user is connected and if he have the teaching staff role or in the assignment staff. + +### Application to our use case + +For our routes we want to test if the user is connected and if he is in the staff of the assignment. So we will +complete the routes definitions like this: +```typescript +registerOnBackend(backend: Express) { + ... + backend.patch('/assignments/:assignmentNameOrUrl/publish', SecurityMiddleware.check(true, SecurityCheckType.ASSIGNMENT_STAFF), this.publishAssignment.bind(this)); + backend.patch('/assignments/:assignmentNameOrUrl/unpublish', SecurityMiddleware.check(true, SecurityCheckType.ASSIGNMENT_STAFF), this.unpublishAssignment.bind(this)); +} +``` + + +## Control the request body + +The Dojo API use the `express-validator` (based on `validator.js`) library to validate and sanitize the request body of +requests. We use the last version of the library that is `7.0.1`. The documentation is available at the following link: +[express-validator](https://express-validator.github.io/docs/). + + +### Application to our use case + +This tutorial will not go deeper in the `express-validator` library because we do not need have request body in our +routes. But if you need to use it, you can find examples in `src/routes/ExerciseRoutes.ts` or +`src/routes/AssignmentRoutes.ts` file (look at the `ExpressValidator.Schema` objects and their usage). + + +## Define the action + +When you define an action you define a function with the following signature: +```typescript +(req: express.Request, res: express.Response): Promise<void> +``` + +To respond to the request you need to use the following method (works even if the user is not connected): +```typescript +return req.session.sendResponse(res: express.Response, code: number, data?: unknown, descriptionOverride?: string, internalCode?: number) +``` +Where : +- `res` is the express response object +- `code` is the HTTP status code of the response +- `data` is the data to send in the response (it must be a JSON serializable value) +- `descriptionOverride` is a field that you can use to give a custom string description of the response (by default it +will be the description of the HTTP status code) +- `internalCode` is the internal code of the response (if you want to add a custom internal code to be more specific +than the HTTP status code) + + +### Application to our use case + +For the implementation we will split our code into four parts: +1. If we want to publish, we check first if it is possible (if the last pipeline status is `success`). +2. Change the project visibility on Gitlab. +3. Change the assignment published status on the Dojo database. +4. Send the response to the client. + +```typescript +// Part 1 +if ( publish ) { + const isPublishable = await SharedAssignmentHelper.isPublishable(req.boundParams.assignment!.gitlabId); + if ( !isPublishable.isPublishable ) { + return req.session.sendResponse(res, StatusCodes.BAD_REQUEST, { lastPipeline: isPublishable.lastPipeline }, isPublishable.status?.message, isPublishable.status?.code); + } +} + +try { + // Part 2 + await GitlabManager.changeRepositoryVisibility(req.boundParams.assignment!.gitlabId, publish ? GitlabVisibility.INTERNAL : GitlabVisibility.PRIVATE); + + // Part 3 + await db.assignment.update({ + where: { + name: req.boundParams.assignment!.name + }, + data : { + published: publish + } + }); + + // Part 4 + req.session.sendResponse(res, StatusCodes.OK); +} catch ( error ) { + if ( error instanceof AxiosError ) { + res.status(error.response?.status ?? HttpStatusCode.InternalServerError).send(); + return; + } + + logger.error(error); + res.status(StatusCodes.INTERNAL_SERVER_ERROR).send(); +} +``` + +## Documentation + +Finally, the fun part. We need to document our new created routes. For that we use the [`OpenAPI` specification in his +3.1.0 version](https://express-validator.github.io/docs/). + +The documentation is yaml formatted and is in the `assets/OpenAPI/OpenAPI.yml` file. + + +## Use case: final code + +### AssignmentRoutes.ts +```typescript +import { Express } from 'express-serve-static-core'; +import express from 'express'; +import { StatusCodes } from 'http-status-codes'; +import RoutesManager from '../express/RoutesManager'; +import SecurityMiddleware from '../middlewares/SecurityMiddleware'; +import SecurityCheckType from '../types/SecurityCheckType'; +import GitlabManager from '../managers/GitlabManager'; +import { AxiosError, HttpStatusCode } from 'axios'; +import logger from '../shared/logging/WinstonLogger'; +import db from '../helpers/DatabaseHelper'; +import GitlabVisibility from '../shared/types/Gitlab/GitlabVisibility'; +import SharedAssignmentHelper from '../shared/helpers/Dojo/SharedAssignmentHelper'; + + +class AssignmentRoutes implements RoutesManager { + registerOnBackend(backend: Express) { + backend.patch('/assignments/:assignmentNameOrUrl/publish', SecurityMiddleware.check(true, SecurityCheckType.ASSIGNMENT_STAFF), this.publishAssignment.bind(this)); + backend.patch('/assignments/:assignmentNameOrUrl/unpublish', SecurityMiddleware.check(true, SecurityCheckType.ASSIGNMENT_STAFF), this.unpublishAssignment.bind(this)); + } + + private async publishAssignment(req: express.Request, res: express.Response) { + return this.changeAssignmentPublishedStatus(true)(req, res); + } + + private async unpublishAssignment(req: express.Request, res: express.Response) { + return this.changeAssignmentPublishedStatus(false)(req, res); + } + + private changeAssignmentPublishedStatus(publish: boolean): (req: express.Request, res: express.Response) => Promise<void> { + return async (req: express.Request, res: express.Response): Promise<void> => { + if ( publish ) { + const isPublishable = await SharedAssignmentHelper.isPublishable(req.boundParams.assignment!.gitlabId); + if ( !isPublishable.isPublishable ) { + return req.session.sendResponse(res, StatusCodes.BAD_REQUEST, { lastPipeline: isPublishable.lastPipeline }, isPublishable.status?.message, isPublishable.status?.code); + } + } + + try { + await GitlabManager.changeRepositoryVisibility(req.boundParams.assignment!.gitlabId, publish ? GitlabVisibility.INTERNAL : GitlabVisibility.PRIVATE); + + await db.assignment.update({ + where: { + name: req.boundParams.assignment!.name + }, + data : { + published: publish + } + }); + + req.session.sendResponse(res, StatusCodes.OK); + } catch ( error ) { + if ( error instanceof AxiosError ) { + res.status(error.response?.status ?? HttpStatusCode.InternalServerError).send(); + return; + } + + logger.error(error); + res.status(StatusCodes.INTERNAL_SERVER_ERROR).send(); + } + }; + } +} + +export default new AssignmentRoutes(); +``` + +### src/types/express/index.d.ts +```typescript +import Session from '../../controllers/Session'; +import { Assignment, Exercise } from '../DatabaseTypes'; + +// to make the file a module and avoid the TypeScript error +export {}; + +declare global { + namespace Express { + export interface Request { + session: Session, + boundParams: { + assignment: Assignment | undefined, exercise: Exercise | undefined + } + } + } +} +``` + +### ParamsCallbackManager.ts +```typescript +import { Express } from 'express-serve-static-core'; +import express from 'express'; +import { StatusCodes } from 'http-status-codes'; +import AssignmentManager from '../managers/AssignmentManager'; + +type GetFunction = (id: string | number, ...args: Array<unknown>) => Promise<unknown> + +class ParamsCallbackManager { + protected listenParam(paramName: string, backend: Express, getFunction: GetFunction, args: Array<unknown>, indexName: string) { + backend.param(paramName, (req: express.Request, res: express.Response, next: express.NextFunction, id: string | number) => { + getFunction(id, ...args).then(result => { + if ( result ) { + this.initBoundParams(req); + (req.boundParams as Record<string, unknown>)[indexName] = result; + + next(); + } else { + req.session.sendResponse(res, StatusCodes.NOT_FOUND, {}, 'Param bounding failed: ' + paramName); + } + }); + }); + } + + initBoundParams(req: express.Request) { + if ( !req.boundParams ) { + req.boundParams = { + assignment: undefined, + exercise : undefined + }; + } + } + + registerOnBackend(backend: Express) { + this.listenParam('assignmentNameOrUrl', backend, (AssignmentManager.get as GetFunction).bind(AssignmentManager), [ { + exercises: true, + staff : true + } ], 'assignment'); + + ... + } +} + +export default new ParamsCallbackManager(); +``` \ No newline at end of file diff --git a/Wiki/Home.md b/Wiki/Home.md index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..e2e52a8c5d8b3b47b4556c2a893abd577c4219cc 100644 --- a/Wiki/Home.md +++ b/Wiki/Home.md @@ -0,0 +1,17 @@ +# Documentation of the `dojo` API + +In this wiki you will find the documentation related to the `dojo` API. + +## Dojo Project +The dojo platform is an online tool built to help practice programming by allowing users to propose assignments and perform them as exercises. + +The two major concepts of the platform are the **assignments** (provided by teaching staff) and the **exercises** (performed by students). + +More details here : [Dojo detailed presentation](0-Dojo-presentation) + + +## Development / Contribution + +* [How to contribute]() - Available soon +* [How to setup your development environment](Development/1-How-to-setup-your-development-environment) +* [How to add a new route](Development/2-How-to-add-a-new-route) \ No newline at end of file