Creating a CI/CD Pipeline for .net from GitLab to AWS Elastic Beanstalk
My AWS Confession
I'm not going to lie, I really love Azure and am not a huge fan of AWS. There I said it and I have my reasons that make sense to me. I realize tons of developers are the opposite, and everyone has their preferences, but there you have it. With that said, I do use AWS from time to time because I have clients on it.
Recently, I had to create a CI/CD pipeline from a GitLab repo to an AWS Elastic Beanstalk. I've made a bunch of pipelines into the Azure world, but this was a new one for me. I wanted a pipeline that handled building, testing, zipping and deploying.
I'm assuming you already created an empty Beanstalk to start with. I just used the AWS plugin in Visual Studio to take care of spinning up the Beanstalk environment and application.
Once you have an environment/application, we first need to create a user in AWS and associate permissions.
1. Open up the IAM (Identity and Access Management) part of AWS.
2. Go to the Users section and Add a user. I called mine "devops-runner".
3. Associate the permissions you need. I used these for my demo purpose (which is overkill), but figure out what the minimal amount of permissions you need is and go from there:
4. Go to the Security credentials tab of the user. Hit the button for "Create access key" and record the Access Key ID and the Secret Access Key. We'll need those in a minute.
5. Create an S3 storage bucket. We'll need that to store the zip file containing the GitLab code as a part of the pipeline.
In GitLab, we need to define a couple variables and put them in the Settings -> CI/CD -> Variables section in GitLab. They are the following:
Note: we'll be using a couple of GitLab's built-in variables as well, these are the following:
Here's my example YAML that you can drop into your CI/CD → Editor in GitLab and customize.
# define our default image image: mcr.microsoft.com/dotnet/sdk:latest variables: # profile the path to your proj file. I usually have more than one project in my solution so this is needed. PROJ_PATH: 'path/to/your/projectname.csproj' RELEASE_FOLDER: 'path/to/your/bin/Release' # define where to put your build. You can make this up. BUILD_FOLDER: '/new/path/to/put/your/builds' # define where to put your zip file containing your build ZIPFILE: '/new/path/to/put/your/builds/$CI_COMMIT_REF_NAME-$CI_PIPELINE_ID.zip' # define some of your AWS variables...these could be defined in the settings/CICD/variables as well if you wanted. REGION: 'us-west-1' APPNAME: 'nice-app-name' ENVNAME: 'nice-env-name' S3BUCKET: 'name-of-s3-bucket' S3KEY: '$CI_COMMIT_REF_NAME-$CI_PIPELINE_ID' #this is what we'll be calling the zip file in S3. # define the stages of the pipeline stages: - build - test - publish - deploy # BUILD the .net project build_job: stage: build only: - master - tags script: - 'dotnet restore $PROJ_PATH' # restore Nuget dependencies - 'dotnet build $PROJ_PATH --configuration Release --no-restore' # build the project artifacts: expire_in: 1 week # save gitlab server space, we copy the files we need to deploy folder later on paths: - '$RELEASE_FOLDER' # saving to deploy folder # TEST the .net project (you are writing tests, right?) test_job: stage: test only: - master - tags script: - 'dotnet test $PROJ_PATH --no-build -l "console;verbosity=detailed"' # PUBLISH the .net project and copy it into your desired build folder (relative to the repo root) publish_job: stage: publish only: - master - tags script: - 'dotnet publish $PROJ_PATH --os linux -o $BUILD_FOLDER' artifacts: expire_in: 1 week # save gitlab server space, we copy the files we need to deploy folder later on paths: - '$BUILD_FOLDER' # saving # DEPLOY to AWS S3 and our beanstalk deploy_job: stage: deploy # let's use one of gitlab's premade images that comes with the aws cli already installed image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest only: - master - tags script: # here's some debugging if you need to see where you're copying your files #- echo $ZIPFILE #- 'echo $BUILD_FOLDER/*' # install zip - apt install zip # go to the build folder and zip everything up - cd $BUILD_FOLDER - zip -r $ZIPFILE * - cd / # configure our AWS security and such - aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID - aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY - aws configure set region $REGION # copy up the build - aws s3 cp $ZIPFILE s3://$S3BUCKET/$S3KEY # using the zip file, update the beanstalk with the latest code - aws elasticbeanstalk create-application-version --application-name $APPNAME --version-label $S3KEY --source-bundle S3Bucket=$S3BUCKET,S3Key=$S3KEY - aws elasticbeanstalk update-environment --application-name $APPNAME --environment-name $ENVNAME --version-label $S3KEY
Hopefully this speeds up your pipeline creation. Happy DevOps!