This article describes how to upgrade an existing Podman Quadlets Installation of Smart ID Certificate Manager (CM).

Prerequisites

• Quadlet installation of Certificate Manager (CM) installed with version CM 8.10.x.

• Podman version 4.9.4 or higher.

• Administrator's Workbench, Registration Authority, and Certificate Controller clients from CM distributable package.

• Access to the new version's distributable package including updated container images ,database migration scripts and configuration files.

Step-by-step instruction

1. Pre-upgrade preparations

1. Ensure database accessibility.
   Verify that the database is accessible and functioning properly before proceeding:

   podman exec -ti <cf-server-container-name> bash # Use JDBC or other database tools to test the connection

2. Stop all running containers.
   Use the following commands to stop the running containers:

   systemctl --user stop cf-server systemctl --user stop pgw

3. Backup the database.
   As part of the backup, make sure to backup the cmdb database, please refer to the relevant database documentation for this step.

4. Verify that the container is stopped.
   Check that all containers are stopped:

   podman ps --all

   Ensure no containers from the previous deployment are running.

5. Backup Volumes: (optional but highly recommended)
   If something goes wrong, the backup can be restored, and the system can return to its previous state.
   Volumes that should be backed up are certs, config, bin and any relevant volume to your installation.

   1. Locate the volumes using:

      podman volume list

      Example (name can change but should contain (cf-server):

      > podman volume list local systemd-cf-server-certs local systemd-cf-server-config local systemd-cf-server-bin local systemd-pgw-config-tomcat local systemd-pgw-config-gw

   2. Locate the Mountpoint on host:

      podman volume inspect <volume-name>

      Example:

      > podman volume inspect systemd-cf-server-certs [ { "Name": "systemd-cf-server-certs", "Driver": "local", "Mountpoint": "/home/azureuser/.local/share/containers/storage/volumes/systemd-cf-server-certs/_data", "CreatedAt": "2024-06-20T07:25:01.386573664Z", "Labels": {}, "Scope": "local", "Options": {}, "MountCount": 0, "LockNumber": 4 } ]

   3. Back up the Mountpoint.

6. Unzip the distribution.

2. Load new container images

1. Import new Images.
   Load the new container images using:

   podman image load --input certificate-manager-<version>/images/cf-server-image-<version>.tar podman image load --input certificate-manager-<version>/images/pgw-image-<version>.tar

2. Verify the Images.
   Ensure the new images are loaded correctly:

   podman images

   Confirm the tags and versions of the new image.

3. Update Quadlet configuration

1. Locate the cf-server.container and pgw.container used for the previous installation.
   The following command displays the path of the files .container under the Loaded attribute.

   systemctl --user status cf-server systemctl --user status pgw

   Example

   > systemctl --user status cf-server × cf-server.service Loaded: loaded (/home/azureuser/.config/containers/systemd/cf-server.container; generated) ...

2. Modify configuration:
   Modify the Image Name and tag to the new Image in the cf-server.container and pgw.container files.

3. Reload systemd daemon:
   Reload the systemd daemon to apply the changes:

   systemctl --user daemon-reload

4. Upgrade database and configuration files

1. Run database scripts. (only necessary for cf-server)
   The upgrade scripts are stored in the upgrade directory under the distributable.
   Execute the database upgrade script towards the database server.

   Ensure the script ran without errors.
   Example for postgres:

   psql -U postgres -d cmdb -f "CMDBUpgrade_PostgreSQL_8_11_0.sql"

2. Update the configuration files.
   Locate the .conf files under the upgrade folder inside the release distribution.

   Open the .conf files and update the configuration files according to the instructions specified by the .conf file. Upgrade should be done towards the volume Mountpoint.
   If the configuration path specified in the .conf file do not map to a volume, changes will automatically be applied hence these files can be ignored.

5. Upgrade Protocol Gateway

6. Restart the containers

1. Start the updated containers:
   Start the new containers using:

   systemctl --user start cf-server systemctl --user start pgw

   Following the logs from startup can be done as follows:

   systemctl --user start cf-server;podman logs -f cf-server

2. Monitor logs.
   Check the logs to ensure the containers started successfully:

   podman logs -f cf-server podman logs -f pgw

7. Post-upgrade validation

1. Verify Connectivity.
   Test connectivity using Administrator's Workbench and/or Registration Authority clients.

2. Verify Configuration.
   Access the container volumes to confirm that the configuration files are correctly applied:

   podman exec -ti cf-server bash ls /opt/cm/server/

3. Check Database Connection.
   Confirm that the container is successfully connected to the upgraded database.

4. Health Checks.
   Enable and test the health check for the Protocol Gateway (pgw):

   systemctl --user start pgw podman logs -f pgw

8. Rollback procedure (if needed)

If any issue occurs during the upgrade, you can roll back using the following steps:

1. Stop the new containers.

   systemctl --user stop cf-server systemctl --user stop pgw

2. Restore volumes.
   Restore the backed-up volumes.

3. Point to the previous Images.
   Point to the old images in cf-server.container and pgw.container.

4. Restore Database

5. Start the old containers.
   Restart the containers using the old quadlets:

   systemctl --user daemon-reload systemctl --user start cf-server systemctl --user start pgw

9. Troubleshooting

• View logs.
  Use the following command to view container logs for debugging:

  podman logs <container-name>

• Inspect container configuration.
  Inspect the container configuration for issues:

  podman inspect <container-name>

• Access shell in container.
  Start a shell inside the container for manual troubleshooting:

  podman exec -ti cf-server bash

• Check systemd status.
  Check the status of the systemd service:

  systemctl --user status cf-server