Multi-job workflows, needs: dependencies, matrix strategy, secrets, caching, artifacts — then the Jenkins equivalent: parallel stages, withCredentials, post conditions, and shared libraries.
name: CI Advanced on: push: branches: [ main, 'feat/**' ] pull_request: branches: [ main ] jobs: # ── Job 1: Lint (runs first, alone) ───────── lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: { node-version: '20', cache: 'npm' } - run: npm ci && npm run lint # ── Job 2: Test on 3 Node versions (PARALLEL) ─ test: runs-on: ubuntu-latest strategy: matrix: node-version: [ 18, 20, 22 ] # 3 parallel jobs fail-fast: false # don't cancel others on 1 fail steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: \${{ matrix.node-version }} cache: 'npm' - run: npm ci && npm test # ── Job 3: Build (waits for lint AND test) ─── build: needs: [ lint, test ] # ← fan-in here runs-on: ubuntu-latest if: github.ref == 'refs/heads/main' steps: - uses: actions/checkout@v4 - run: npm ci && npm run build
needs: [lint, test] makes the build job wait until all specified jobs pass. If any one fails, build is skipped.
matrix: { node: [18,20,22] } → 3 jobsmatrix: { os: [ubuntu, windows], node: [18,20] } → 4 jobs (all combos)include: — add extra variables to a specific comboexclude: — skip a specific combofail-fast: false — don't cancel siblings if one fails# === Set secrets in GitHub === # Repo → Settings → Secrets → Actions → New # DOCKER_USERNAME, DOCKER_PASSWORD, APP_VERSION # === Workflow-level env (all jobs see this) === env: NODE_ENV: production APP_NAME: my-devops-app jobs: deploy: runs-on: ubuntu-latest # Job-level env (this job only) env: DOCKER_USER: \${{ secrets.DOCKER_USERNAME }} steps: - name: Login to Docker Hub uses: docker/login-action@v3 with: username: \${{ secrets.DOCKER_USERNAME }} password: \${{ secrets.DOCKER_PASSWORD }} - name: Build and push image run: | docker build -t \${{ secrets.DOCKER_USERNAME }}/\${{ env.APP_NAME }}:\${{ github.sha }} . docker push \${{ secrets.DOCKER_USERNAME }}/\${{ env.APP_NAME }}:\${{ github.sha }} # WRONG — never do this! # - run: echo "Token is ${{ secrets.DOCKER_PASSWORD }}" # === Environment-scoped secrets (prod vs staging) === jobs: deploy-prod: environment: production # uses prod-specific secrets runs-on: ubuntu-latest steps: - run: echo "Deploying to production..."
Precedence: Environment > Repository > Organisation
***. But partial values, encoded forms, or derived values may still leak. Never echo/print secret values. Use them only in with: or env: blocks.
\$\{{ github.sha }} — commit SHA (use as Docker image tag)\$\{{ github.ref_name }} — branch or tag name\$\{{ github.actor }} — who triggered the run\$\{{ github.run_number }} — incremental build number
# === Method 1: cache: 'npm' in setup-node (easiest) === steps: - uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' # ← caches ~/.npm automatically - run: npm ci # uses cache on 2nd run → 60s → 5s # === Method 2: actions/cache (more control) === steps: - name: Cache node_modules uses: actions/cache@v4 with: path: ~/.npm key: npm-\${{ runner.os }}-\${{ hashFiles('**/package-lock.json') }} restore-keys: npm-\${{ runner.os }}- - run: npm ci # Cache key uses the package-lock.json hash: # - Same lock file → cache hit (fast) # - Lock file changed → cache miss → fresh install
jobs: build: steps: - run: npm run build # creates dist/ - uses: actions/upload-artifact@v4 with: name: dist-files path: dist/ retention-days: 7 deploy: needs: build steps: - uses: actions/download-artifact@v4 with: name: dist-files - run: ls dist/ # available here!
The cache key determines when to use cached data vs fresh:
hashFiles('**/package-lock.json') — cache busted when lock file changesrunner.os — separate caches per OSrestore-keys: — fallback to partial match (slightly stale cache beats full miss)pipeline { agent any tools { nodejs 'NodeJS-20' } environment { APP_NAME = 'my-devops-app' } stages { stage('Checkout') { steps { checkout scm } } // Parallel: Lint + Test run simultaneously stage('Quality Checks') { parallel { stage('Lint') { steps { sh 'npm ci && npm run lint' } } stage('Test') { steps { sh 'npm ci && npm test' } post { always { junit '**/test-results/*.xml' } } } } } // Only build on main branch stage('Build Docker') { when { branch 'main' } steps { withCredentials([usernamePassword( credentialsId: 'docker-hub-creds', usernameVariable: 'DOCKER_USER', passwordVariable: 'DOCKER_PASS' )]) { sh """ docker login -u $DOCKER_USER -p $DOCKER_PASS docker build -t $DOCKER_USER/$APP_NAME:$BUILD_NUMBER . docker push $DOCKER_USER/$APP_NAME:$BUILD_NUMBER """ } } } // Human approval gate before production stage('Approve Production') { when { branch 'main' } steps { input message: 'Deploy to production?', ok: 'Deploy' } } stage('Deploy') { when { branch 'main' } steps { sh 'echo "Deploying $APP_NAME:$BUILD_NUMBER"' } } } post { always { echo 'Pipeline finished' } success { slackSend '✅ Build passed' } failure { slackSend '❌ Build failed — check logs' } cleanup { cleanWs() } // delete workspace } }
parallel { stage('a') stage('b') } — run stages simultaneously (= GHA jobs in parallel)when { branch 'main' } — conditional stage — only runs on main (= GHA if: github.ref == 'refs/heads/main')withCredentials([...]) — inject credentials securely into steps (= GHA secrets.*)input — pauses pipeline until a human clicks Approve (= GHA Environments with required reviewers)post { always/success/failure } — runs after pipeline regardless of resultcleanWs() — clean workspace plugin — avoids stale fileswithCredentials injects them as environment variables only inside its block — they're masked in logs and not accessible outside the block. Equivalent of GitHub's secrets.*.
10 teams have the same "build Docker + push + deploy" logic in their Jenkinsfiles. One change needs to be made in 10 places. Solution: Shared Library.
// vars/buildAndPush.groovy (in shared-lib repo) def call(Map config) { sh "docker build -t ${config.image}:${config.tag} ." sh "docker push ${config.image}:${config.tag}" } // Any Jenkinsfile using the library: @Library('jenkins-shared-lib') _ pipeline { agent any stages { stage('Build') { steps { buildAndPush( image: 'myorg/myapp', tag: env.BUILD_NUMBER ) } } } }
# .github/workflows/reusable-build.yml on: workflow_call: # ← makes it reusable inputs: image-name: required: true type: string jobs: build: runs-on: ubuntu-latest steps: - run: docker build -t \${{ inputs.image-name }} . # Caller workflow in any repo: jobs: build-app: uses: org/shared-workflows/.github/workflows/reusable-build.yml@main with: image-name: myorg/myapp
jenkins-shared-lib/ (separate Git repo)
├── vars/
│ ├── buildAndPush.groovy ← global functions
│ └── runTests.groovy
├── src/
│ └── org/helper/Utils.groovy
└── resources/
└── config.yaml
Register in Jenkins: Manage Jenkins → System → Global Pipeline Libraries → add repo URL + name.
npm ci in pipelines: installs exact locked versions from package-lock.json, fails if lock file is stale (catches drift), never modifies lock file, faster than npm install in CI. npm install is for development; npm ci is for CI.
Multi-job GHA with matrix + secret → Jenkins equivalent with parallel stages + credentials
.github/workflows/ci-advanced.yml with 3 jobs: lint, test (matrix: Node 18 + 20), build (needs: both). Verify 4 jobs run in the Actions tab (lint + test×2 parallel + build).APP_VERSION=1.0.0. Reference it in the build job with \$\{{ secrets.APP_VERSION }}. Verify it appears as *** in logs.cache: 'npm' to every actions/setup-node step. Trigger the workflow twice. Compare run times — second run should be ~60s faster.Jenkinsfile with a parallel { } block for lint + test. Add when { branch 'main' } to the build stage. Add a post { success/failure } block.app-version-creds. Use withCredentials in the Jenkinsfile build stage to inject it.name: CI Advanced on: push: branches: [ main, 'feat/**' ] pull_request: branches: [ main ] jobs: # ── 1. Lint ───────────────────────── lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: { node-version: '20', cache: 'npm' } - run: npm ci - run: npm run lint || echo "No lint script — OK" # ── 2. Test — matrix Node 18 + 20 ─── test: runs-on: ubuntu-latest strategy: matrix: node-version: [ 18, 20 ] fail-fast: false steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: \${{ matrix.node-version }} cache: 'npm' - run: npm ci && npm test # ── 3. Build (needs lint + test) ───── build: needs: [ lint, test ] runs-on: ubuntu-latest if: github.ref == 'refs/heads/main' env: APP_VERSION: \${{ secrets.APP_VERSION }} steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: { node-version: '20', cache: 'npm' } - run: npm ci - name: Build run: echo "Building version $APP_VERSION" - name: Upload dist uses: actions/upload-artifact@v4 with: name: build-\${{ github.sha }} path: dist/ if-no-files-found: warn
pipeline { agent any tools { nodejs 'NodeJS-20' } stages { stage('Checkout') { steps { checkout scm } } // Parallel lint + test (= GHA parallel jobs) stage('Quality') { parallel { stage('Lint') { steps { sh 'npm ci' sh 'npm run lint || echo "No lint"' } } stage('Test') { steps { sh 'npm ci && npm test' } } } } stage('Build') { when { branch 'main' } steps { withCredentials([ string(credentialsId: 'app-version', variable: 'APP_VERSION') ]) { sh """ echo "Building $APP_VERSION" npm ci """ } } } } post { success { echo '✅ Build passed!' } failure { echo '❌ Build failed!' } always { cleanWs() } } }
ci: add multi-job pipeline with matrix and parallel stages
3 questions · 5 minutes · matrix, secrets, Jenkins credentials
needs: [lint, test] do in a job definition?npm ci instead of npm install?\$\{{ secrets.TOKEN }} for injecting credentials securely?environment { TOKEN = 'hardcoded-value' }withCredentials([string(credentialsId: 'token-id', variable: 'TOKEN')])sh 'cat /etc/jenkins/secrets/TOKEN'params.TOKENci: add multi-job pipeline with matrix ✓# Conditional on branch + event if: github.ref == 'refs/heads/main' && github.event_name == 'push' # Continue even if a step fails - run: npm run lint continue-on-error: true # Set output from one step, use in next - id: version run: echo "tag=$(git describe --tags)" >> $GITHUB_OUTPUT - run: echo "Version is \${{ steps.version.outputs.tag }}" # Timeout on a step - run: npm test timeout-minutes: 10 # Retry a flaky step - uses: nick-fields/retry@v3 with: timeout_minutes: 5 max_attempts: 3 command: npm test # Environment-gated deployment jobs: deploy-prod: environment: name: production url: https://myapp.com # GitHub will show approval UI before this runs
// Run stage only on specific branch when { anyOf { branch 'main' branch 'release/*' } } // Environment variables from credentials withCredentials([ usernamePassword( credentialsId: 'docker-hub', usernameVariable: 'DOCKER_USER', passwordVariable: 'DOCKER_PASS'), string( credentialsId: 'slack-token', variable: 'SLACK_TOKEN') ]) { sh 'docker login -u $DOCKER_USER -p $DOCKER_PASS' } // Parameterised build (human input at trigger) parameters { choice( name: 'ENVIRONMENT', choices: ['staging', 'production'], description: 'Deploy target') booleanParam( name: 'SKIP_TESTS', defaultValue: false) } // Use: params.ENVIRONMENT // Archive build artifacts post { success { archiveArtifacts artifacts: 'dist/**' publishHTML(target: [ reportDir: 'coverage', reportFiles: 'index.html', reportName: 'Coverage Report' ]) } }
| Concept | GitHub Actions | Jenkins |
|---|---|---|
| Pipeline file | .github/workflows/ci.yml | Jenkinsfile |
| Top-level block | jobs: | pipeline { stages { } } |
| Execution unit | job | stage('Name') |
| Runner/Agent | runs-on: ubuntu-latest | agent any / agent { label 'x' } |
| Shell command | run: npm test | sh 'npm test' |
| Secrets | \$\{{ secrets.NAME }} | withCredentials([...]) |
| Parallel | jobs run in parallel by default | parallel { stage('a') stage('b') } |
| Sequential | needs: [job-a] | stages execute top-to-bottom by default |
| Conditional | if: github.ref == 'refs/heads/main' | when { branch 'main' } |
| Approval gate | environment: production (required reviewers) | input 'Deploy to prod?' |
| Post actions | if: always() / failure() / success() | post { always { } failure { } } |
| Reusable | workflow_call + uses: org/repo/wf@main | @Library('shared-lib') + function call |
| Env vars | env: at workflow/job/step level | environment { VAR = 'val' } |
input).myapp:\$\{{ github.sha }}. This gives full traceability — you always know exactly which code is running in production. Semantic version tags (v1.2.3) for releases.
| Problem | Cause | Fix |
|---|---|---|
| Workflow not triggering | YAML syntax error or wrong on: branch pattern | Use yamllint or GitHub's YAML validator. Check branch pattern matches. |
| npm ci fails "missing package-lock.json" | package-lock.json not committed | Run npm install locally, commit the lock file. Never gitignore package-lock.json. |
| Secret shows as "***" in unexpected place | Normal — GitHub redacts known secret values | Expected behaviour. Never intentionally print secrets. |
| matrix job fails on one version only | Version-specific API or syntax difference | Click the failing version in Actions tab. Add fail-fast: false to see all results. |
| build job skipped (yellow ○) | needs: job failed OR if: condition not met | Check if: condition. If needs job failed, fix that first. Run on correct branch. |
| cache always missing (no speedup) | Cache key changing every run | Key must include stable part (os) + changing part (lock file hash). Use restore-keys: for fallback. |
| Jenkins: "No such tool" NodeJS | NodeJS tool not configured or name mismatch | Manage Jenkins → Tools → NodeJS. Name must exactly match Jenkinsfile tools { nodejs 'NAME' }. |
| Jenkins: parallel stages not running | Only 1 agent available | Parallel requires separate agents. Docker agent or add more Jenkins agents. |
| withCredentials secret is empty | credentialsId doesn't match stored credential ID | Manage Jenkins → Credentials → check exact ID value. IDs are case-sensitive. |
| Day | Topic | Lab Output | Status |
|---|---|---|---|
| Day 11 | CI/CD Concepts | First pipeline live in Actions tab | ✅ |
| Day 12 ← TODAY | Pipelines Deep Dive | Multi-job GHA + Jenkins parallel stages | ✅ |
| Day 13 | Testing in CI | Jest + 80% coverage gate | Tomorrow |
| Day 14 | Artifacts & Docker | Docker image pushed to GHCR on merge | Thu |
| Day 15 | Continuous Deployment | Staging auto + prod approval gate | Fri |
my-devops-app/ ├── .github/ │ └── workflows/ │ ├── ci.yml ← Day 11 (basic) │ ├── ci-advanced.yml ← Day 12 (matrix) │ ├── test.yml ← Day 13 (Jest) │ ├── docker.yml ← Day 14 (Docker) │ └── deploy.yml ← Day 15 (CD) └── Jenkinsfile ← Days 11-12
# npm install (development) npm install # → Reads package.json # → Resolves versions (may get newer patch) # → Updates package-lock.json if needed # → Merges with existing node_modules # → Fine for developer machines # npm ci (CI/production — always use this) npm ci # → Reads package-lock.json only # → Installs EXACT versions (no resolution) # → Fails if package-lock.json is missing # → Fails if package-lock.json is out of sync with package.json # → Deletes node_modules FIRST (clean install) # → Never modifies package-lock.json # → FASTER in CI (no resolution step) # === The lock file sync problem === # Developer adds package, forgets to commit lock file # npm install: silently resolves → works # npm ci: FAILS → catches the problem ← correct behaviour # === Fix: always commit package-lock.json === git add package.json package-lock.json git commit -m "chore: add express dependency"
npm ci with cache ~5 sec vs cold 60+ secactions/setup-node cache: 'npm'yarn.lock → yarn install --frozen-lockfilepnpm-lock.yaml → pnpm install --frozen-lockfilerequirements.txt → pip install -r requirements.txt# Matrix strategy: matrix: node: [ 18, 20 ] fail-fast: false steps: - uses: actions/setup-node@v4 with: node-version: \${{ matrix.node }} cache: 'npm' # Fan-in build: needs: [ lint, test ] if: github.ref == 'refs/heads/main' # Secrets env: TOKEN: \${{ secrets.MY_TOKEN }} # Artifact upload - uses: actions/upload-artifact@v4 with: name: dist path: dist/ # Conditional step - if: failure() run: echo "Step failed!"
// Parallel stages stage('Quality') { parallel { stage('Lint') { steps { sh '...' } } stage('Test') { steps { sh '...' } } } } // Conditional stage stage('Deploy') { when { branch 'main' } steps { sh 'deploy.sh' } } // Credentials withCredentials([ usernamePassword( credentialsId: 'docker-hub', usernameVariable: 'USER', passwordVariable: 'PASS') ]) { sh 'docker login -u $USER -p $PASS' } // Approval gate input message: 'Deploy to prod?', ok: 'Go!' // Post conditions post { success { echo '✅' } failure { echo '❌' } always { cleanWs() } }
npm ci in pipelines. npm install on developer machines. Commit package-lock.json. Never gitignore it.