Skip to main content

How-To: Backup & Restore InfluxDB Databases with TrilioVault for Kubernetes


In this blog post, you will find out how to backup and restore InfluxDB time-series databases to S3 compatible or NFS targets.

InFluxDB

InfluxDB is an open source time-series database developed by InfluxData. It is optimized for fast, high-availability storage and retrieval of time-series data for use with operations monitoring, application metrics, Internet of Things sensor data, real-time analytics and more. InfluxDB is designed to handle high write and query loads.

Why Backup?

Data loss is a serious problem. Losing files means losing time and money while you restore or recover information essential to the business. In the case of a server failure or outage, valuable data could be lost from the database forever. Losing files and documents often has a lasting impact on the company’s financial health. Consequently, backup and recovery are essential to the overall viability of the business.

The dynamic architecture of Kubernetes puts a lot of demands on a backup and recovery solution, so finding the right one can be difficult. In Kubernetes, applications are microservices-based and dynamic in nature. Applications can be deployed using labels, Helm charts or Operators. There are also multiple namespaces and resources that are either tied to a specific namespace or common across all namespaces. As such, metadata is vital for application-centric backup.

Considering all of the above, you need to identify software that can provide an application-centric, cloud-native data protection solution. TrilioVault for Kubernetes (TVK) offers a platform that satisfies these requirements.

The image below provides a quick overview of how backup works in TVK:

Now, take a look at how TVK performs application consistent backup and restore of InfluxDB.

Pre-requisite: Install TrilioVault for Kubernetes (TVK) and configure using the following steps:

  • Install Test CSI Driver — Leverage the test hostpath driver if your environment does not support the driver with Snapshot capabilities today.
  • Software Access and Installation — Access software and install it based on the specific directions for your environment.
  • License — Leverage the Free Trial or Free Basic Edition by following instructions from the licensing page. If you already have an enterprise license, you can skip this step.

Create a target where backups will be stored

The Backup Target CRD specifies the backup storage media. TrilioVault supports either AWS S3 compatible object storage or NFS. All the backups created will be saved on the backup target specified in the Application CR spec. An Amazon S3 target example is provided below:

apiVersion: triliovault.trilio.io/v1
kind: Target
metadata:
  name: demo-s3-target
spec:
  type: ObjectStore
  vendor: AWS
  objectStoreCredentials:
    url: "https://s3.amazonaws.com"
    accessKey: "AaBbCcDdEeFf"
    secretKey: "BogusKeyEntry"
    bucketName: "S3_Bucket_US_East"
    region: "us-east-1"
  thresholdCapacity: 1000Gi

$ kubectl create -f tv-backup-target.yaml

InfluxDB databases can be deployed on Kubernetes using Helm charts. In the example below, a new release called my-release is created from helm chart bitnami/influxdb using the helm install command.

$ helm repo add bitnami https://charts.bitnami.com/bitnami
$ helm repo update
$ helm install my release bitnami/influxdb

Add Test Data

Let’s add some test data to the InfluxDB database. Please follow the commands below to create a test database, table and records.

$ Root$ ocexec -it influxdb-deployment-54c5c78664-f7z7n -- influx
Connected to http://localhost:8086 version 1.7.4
InfluxDB shell version 1.7.4
Enter an InfluxQL query
>
>auth
username: xxxx
password: xxxx
>
>CREATE DATABASE pirates
>use pirates
Using database pirates
>INSERT treasures,captain_id=dread_pirate_roberts value=801
>INSERT treasures,captain_id=flint value=29
>INSERT treasures,captain_id=sparrow value=38
>INSERT treasures,captain_id=tetra value=47
>INSERT treasures,captain_id=crunch value=109
>
>SELECT * FROM treasures
name:treasures
time                captain_id          value
----                ----------          -----
1613634634180857624 dread_pirate-roberts 801
1613634654035942732 flint                29
1613634666125641798 sparrow              38
1613634676936723375 tetra                47
1613634694340402128 crunch               109
>
>exit

Create Hooks

Hooks enable the ability to inject commands into pods or containers before and after a backup via pre/post commands. As a result, customers can take application consistent backups of their application.

  • InfluxDB has its own built-in backup and restore capabilities. Their scope can range from individual databases to shards, policies, etc. A backup creates a copy of the metastore and shard data at a point-in-time and stores the copy in the specified directory.
  • A full backup creates a copy of the metastore and shard data.
  • An incremental backup creates a copy of whatever metastore and shard data has changed since the last incremental backup.
  • If there are no existing incremental backups, the system automatically performs a complete backup.
  • Use the InfluxDB backup command inside the TVK backup hook (see below).

apiVersion: triliovault.trilio.io/v1
kind: Hook
metadata:
  name: influx-hook
spec:
  pre:
    execAction:
      command:
        - "bash"
        - "-c"
        - "bkpfile=/tmp/snapdate +%Y%m%dT%H%M; influxdbackup $bkpfile"
    ignoreFailure: false
    maxRetryCount: 1
    timeoutSeconds: 10
  post:
    execAction:
      command:
        - "bash"
        - "-c"
        - "echo 'post hook action completed'"
    ignoreFailure: false
    maxRetryCount: 1
    timeoutSeconds: 10

$kubectl create -f tv-influx-hook.yaml

Create BackupPlan

The BackupPlan specifies the backup job. The specification includes the backup schedule, backup target and the resources to backup.

apiVersion: triliovault.trilio.io/v1
kind: BackupPlan
metadata:
name: influx-hook-label-backup-plan
spec:
backupNamespace: default
backupConfig:
  target:
    name: demo-s3-target
backupPlanComponents:
  custom:
- matchLabels:
app:influxdb
hookConfig:
  mode: Sequential
  hooks:
    - hook:
        name: influx-hook
      podSelector:
        labels:
          - matchLabels:
              app: influxdb
        regex: influxdb*

$kubectl create -f tv-influx-backupplan.yaml

Create Backups
The backup takes either a full or incremental backup of the resources specified in the BackupPlan spec. The first backup of the application will always be a full backup, even if the user specifies their backup type as incremental.

apiVersion: triliovault.trilio.io/v1
kind: Backup
metadata:
  name: influx-hook-label-full-backup
spec:
  type: Full
  scheduleType: Periodic
  backupPlan:
    name: influx-hook-label-backup-plan

$kubectl create -f tvk-mysql-backup.yaml

The Restore

The restore specifies which backup resources should be restored. Resources can be restored to the same namespace or to a different one. Let’s take a look at restoring to a different namespace, e.g. “restore-ns.” This needs to be created if it does not already exist.

apiVersion: triliovault.trilio.io/v1
kind: Restore
metadata:
  name: influx-label-restore
spec:
  backupPlan:
    name: influx-hook-label-backup-plan
  source:
    type: Backup
    backup:
      name: influx-hook-label-full-backup
    target:
      name: demo-s3-target
  restoreNamespace: restore-ns
  skipIfAlreadyExists: true

$kubectl create -f tv-influx-restore.yaml

Conclusion

When you use this procedure, backing up a time-series database like InfluxDB with TrilioVault for Kubernetes is as easy as it gets. The backups are application-consistent, and the “hooks” allow the user to perform any pre/post backup actions. Furthermore, TrilioVault for Kubernetes provides a wide range of helpful features, including:

  • Native Kubernetes application
  • Stores metadata and all application resources to a specified target
  • Supports application deployment types of Helm/Label/Operators and S3 or NFS-based backup targets
  • Provides application hooks to ensure data consistent backups

Altogether, TVK provides a strong platform for enterprise database backups.

For more information on TrilioVault for Kubernetes request a demo or download your free trial today.

Sachin Kulkarni, Solutions Engineer for Trilio