Set up multiple configuration files for a project
7 days ago
A CircleCI project integrated with GitHub can have one or more pipelines, each defined by a separate configuration file. CircleCI configuration files can be stored in the same repo as your code, or even in different repos if you wish.
Each pipeline can be configured to run on one or many specific trigger events.
If your existing configuration file is getting too large to manage, and it includes several unrelated workflows, consider splitting it into more than one file.
Using multiple pipelines and triggers, you can build different pipelines for different events, For example, you could have:
-
A
build-testpipeline that runs on every push. Select "All pushes". -
A
e2e-testspipeline, longer and more costly, that runs once when the PR is taken out of draft. Select "PR marked ready for review". -
A
teardown-envpipeline that runs when a PR is merged. Select "PR merged".
For a full list, see the GitHub trigger event options page.
Prerequisites
To use multiple pipelines and configs for a project in CircleCI, you will need to meet the following prerequisites:
-
A CircleCI account connected to your code in GitHub. You can sign up for free. Any of CircleCI’s GitHub integrations are supported, read our community forum for background on how this functionality can be used with an org integrated CircleCI’s GitHub OAuth App. To use multiple pipelines for a project you will use the CircleCI GitHub App integration.
-
A project set up in CircleCI that you want to configure with multiple, distinct pipelines defined with different configuration files. See the Create a project in CircleCI page for steps to get your project set up in CircleCI.
1. Add additional configuration files to your project repository
To use more than one configuration for your project you will create more than one pipeline. You can set up separate triggers for each pipeline. Each pipeline is defined by a YAML config file.
In this first step, create the configs you need to build your project. Config files can be stored anywhere you choose within your GitHub org. This can be in the same repo, or a different repo. Your configuration files can have any name you choose, but must have the .yml extension.
If your project is already set up in CircleCI, a configuration file will likely exist already. If you do not see a .circleci directory containing a YAML config, check which branch you are on. You may have set this up on a non-default branch during the set up process.
Each of your configuration files represents the full set of commands necessary for a pipeline to execute so it should be complete with all necessary elements, for example, version, jobs, workflows, commands, executors, orbs etc.
| Splitting up an existing configuration? If you are following these steps because you have an existing project building on CircleCI and you want to split your configuration into multiple files, you can create the config files you need and move over parts of your existing configuration file to your new YAML files. |
2. Set up a pipeline
Next, add a new pipeline for each config you have. Create and edit pipeline in your .
-
In the CircleCI web app, select your org from the org cards on your user homepage.
-
Select Projects from the sidebar and locate your project from the list. You can use the search to help.
-
Select the ellipsis
next to your project and select Project Settings.You can also access project settings from each project overview page using the Settings button. -
Select Project Setup in the sidebar.
-
Select Add Pipeline.
If you have not installed the CircleCI GitHub App into your GitHub organization you will be prompted to do so. Installing the GitHub App is a one-time step that can be done by an organization admin. Once installed, further GitHub App pipelines can be setup without this installation step. For a guide to using both OAuth and GitHub App integrations, see the Using the CircleCI GitHub App in an OAuth organization page. If you run into issues when installing the GitHub App, for example, App looks installed on GitHub, but does not show up on CircleCI, submit a request through this form. -
Complete the required form fields:
-
Give your pipeline a descriptive name.
-
Under Config Source select the repository where the config file is stored. This can be the same repository as your code or a different repository.
-
Specify where your config file is located under Config File Path.
-
Select your code source under Checkout Source — this is the code that will be checked out when using the
checkoutstep in your config file.
-
-
Select Save
3. Create a new trigger
Next, create a trigger for each pipeline you have created for your project. Alternatively, you can trigger a pipeline manually via the web app.
-
Following from the previous step, select GitHub trigger + at the bottom of the pipeline box.
-
Choose when to run the pipeline by selecting an option from the Event menu.
-
Select the Event Source repository from the dropdown. This defaults to the config source repository. If you change it to a repository that does not match the pipeline’s config or checkout source, you will be prompted to specify a fixed branch to be used for fetching config and checking out code.
-
Select Save