Skip to content
Snippets Groups Projects
Commit 7a21292b authored by francisc.mendonca's avatar francisc.mendonca
Browse files
parents d29854eb a5c612fa
Branches
No related tags found
No related merge requests found
KubernetesClass.png

59.1 KiB

...@@ -4,7 +4,7 @@ ...@@ -4,7 +4,7 @@
In this exercise, students will deploy a Kubernetes cluster locally to manage an application that retrieves and stores electrical consumption data, forecasts future consumption, and presents both historical and projected consumption trends. In this exercise, students will deploy a Kubernetes cluster locally to manage an application that retrieves and stores electrical consumption data, forecasts future consumption, and presents both historical and projected consumption trends.
The electrical consumption is reprsented by a a CSV file stored on S3. This CSV file has 11 columns. The first column is the time stamp. The other ten columns each represent the power measurements (P) of a smart meter's electricity consumption. Measurements are taken every 15 minutes. A row in the CSV file therefore corresponds to the power measurement at a given time t (HH:00, HH:15, HH:30. HH:45) for the 10 smart meters. The measures cover the period 04.01.2021 - 31.12.2022. The electrical consumption is reprsented by a a CSV file stored on S3. This CSV file has 11 columns. The first column is the time stamp. The other ten columns each represent the power measurements (P) of a smart meter's electricity consumption. Measurements are taken every 15 minutes. A row in the CSV file therefore corresponds to the power measurement at a given time t (HH:00, HH:15, HH:30. HH:45) for the 10 smart meters. The measures cover the period 01.01.2021 - 31.05.2022.
The application will be deployed on a local kubernetes cluster created using the [kind] (https://kind.sigs.k8s.io/) tool. The application will be deployed on a local kubernetes cluster created using the [kind] (https://kind.sigs.k8s.io/) tool.
...@@ -34,11 +34,7 @@ Redis is an in-memory database largely used as a cache. In this case, we’ll us ...@@ -34,11 +34,7 @@ Redis is an in-memory database largely used as a cache. In this case, we’ll us
Grafana is an analytics visualization platform which will be used to visualize the historical and forecasted data being processed. It connects to Redis and displays the RedisTimeSeries as a line graph, showing passed and future power consumption. Grafana is an analytics visualization platform which will be used to visualize the historical and forecasted data being processed. It connects to Redis and displays the RedisTimeSeries as a line graph, showing passed and future power consumption.
## Composition ## Setup
## Steps
### Setup
1. Clone this repository and create an account on [Docker Hub](https://hub.docker.com). 1. Clone this repository and create an account on [Docker Hub](https://hub.docker.com).
2. Install [Docker](https://docs.docker.com/engine/install/) and [Kind](https://kind.sigs.k8s.io/docs/user/quick-start/). If you are installing Kind on Mac, use brew tool: brew install kind. The Installation instructions on MacOS available on [this page](https://kind.sigs.k8s.io/docs/user/quick-start/#installing-from-release-binaries) do not work. 2. Install [Docker](https://docs.docker.com/engine/install/) and [Kind](https://kind.sigs.k8s.io/docs/user/quick-start/). If you are installing Kind on Mac, use brew tool: brew install kind. The Installation instructions on MacOS available on [this page](https://kind.sigs.k8s.io/docs/user/quick-start/#installing-from-release-binaries) do not work.
4. Create a [Kind configuration file](https://kind.sigs.k8s.io/docs/user/quick-start/#multi-node-clusters) composed of one control-plane node, and 5 worker nodes 4. Create a [Kind configuration file](https://kind.sigs.k8s.io/docs/user/quick-start/#multi-node-clusters) composed of one control-plane node, and 5 worker nodes
...@@ -73,11 +69,11 @@ service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h37m ...@@ -73,11 +69,11 @@ service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h37m
You are ready to deploy services to your cluster. You are ready to deploy services to your cluster.
### Deployment Files ## Deployment Files
Kubernetes services can be deployed using Deployment Files, `YAML` files that describe how and where a service should be deployed. In this repositiory you have a folder called `code` which has the following structure: Kubernetes services can be deployed using Deployment Files, `YAML` files that describe how and where a service should be deployed. In this repositiory you have a folder called `deployment` which has the following structure:
```bash ```bash
|- code/ |- deployment/
|---- data-retreival/ |---- data-retreival/
|---- forcast/ |---- forcast/
|---- data-retrieval-deployment.yaml |---- data-retrieval-deployment.yaml
...@@ -86,9 +82,7 @@ Kubernetes services can be deployed using Deployment Files, `YAML` files that de ...@@ -86,9 +82,7 @@ Kubernetes services can be deployed using Deployment Files, `YAML` files that de
|---- redis-deployment.yaml |---- redis-deployment.yaml
``` ```
The `data-retrieval-deployment.yaml` and `forecast-deployment.yaml` files are to be completed. The yaml files are incomplete and need to be completed before any deployments.
### Code
Both the `data-retrieval` and `forecast` folders have the following structure: Both the `data-retrieval` and `forecast` folders have the following structure:
...@@ -99,48 +93,47 @@ Both the `data-retrieval` and `forecast` folders have the following structure: ...@@ -99,48 +93,47 @@ Both the `data-retrieval` and `forecast` folders have the following structure:
|---- Dockerfile |---- Dockerfile
``` ```
## Deployment
The application needs to be deployed in the following order:
1. Redis + Grafana
2. Data Retrieval - It needs to finish before the next step
3. Forecast
If the order isn't followed, there will be several errors happening. `Redis` needs to be the first to be deployed as both the `Data-Retrieval` and `Forecast` are dependant on it.
## Tasks ## Tasks
### Task 1: redis deployment ### Task 1: Redis deployment
1. Fill the redis-deployment.yaml
2. Dploy the redis module using kubectl.
### Task 2: Data Retrieval deployment ### Task 2: Data Retrieval deployment
Read the data-retrieval-deployment.yaml carefully and spot the name of the secrets used. This secrets must be generated by this command: The data-retrieval module must access the S3 object storage to read the CSV file containing the electrical consumption. You must therfore use the AWS Acess key and secret key. We use "secrets" to propagate confidential data through the cluster. Read the data-retrieval-deployment.yaml carefully and spot the name of the secrets used. The secrets must be generated by the following command:
```bash ```bash
kubectl create secret generic <name-of-the-secrets> \ kubectl create secret generic <name-of-the-secrets> \
--from-literal=AWS_ACCESS_KEY_ID=<Your access key> \ --from-literal=AWS_ACCESS_KEY_ID=<Your access key> \
--from-literal=AWS_SECRET_ACCESS_KEY=<your secret key> --from-literal=AWS_SECRET_ACCESS_KEY=<your secret key>
``` ```
Build the Data Retrieval docker, complete the file "data-retrieval-deployment.yaml" and deploy data-retrieval module. 1. Build the Data Retrieval docker and push it to your dockerhub account
2. complete the file "data-retrieval-deployment.yaml"
3. Deploy the data-retrieval module using kubectl
### Task 3: Forecast deployment ### Task 3: Forecast deployment
1. Build the container and push it to your dockerhub account
2. Fill the forecast-deployment.yaml
3. Deploy the forecast module using kubectl. Keep in mind tha data-retrieval deployment must finish its execution before deploying the forecast module. Use "kubectl logs" to ensure that the data-retrieval module has finished.
### Task 4: Grafana deployment ### Task 4: Grafana deployment
Fill the grafane-deployment.yaml and deploy.
### Task 5: Load Balancer ### Task 5: Load Balancer
Kubernetes provides Load Balancing capabilities natively. This permits the distribution of load between several replicas of the same pod. But, as we are deploying using kind, to access this ability we have to create a port-forward between the service and the host device: Kubernetes provides Load Balancing capabilities natively. This permits the distribution of load between several replicas of the same pod. But, as we are deploying using kind, to access this ability we have to create a port-forward between the service and the host device:
```bash ```bash
kubectl port-forward service/grafana 3000:3000 kubectl port-forward service/grafana-service 3000:3000
``` ```
In this case we only require one port forward, between the grafana service (the frontend to this service), and the host. Once this PF is established, we are able to access the grafana interface on `http://localhost:3000`. In this case we only require one port forward, between the grafana service (the frontend to this service), and the host. Once this PF is established, we are able to access the grafana interface on `http://localhost:3000`.
## Grafana Dashboard Configurations ## Task 6: Grafana Dashboard Configuration
Build the dashboard by copying the following photos: Build the dashboard by copying the following photos:
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment