Performance optimization

If you intend to use CTFreak intensively (e.g. keep a history of more than 400,000 executions and/or launch more than 400,000 executions per day), here are some tips you can apply in priority order.

(Note that executing the same shell script on 10 nodes counts as a single execution.)

Memory usage

Generally speaking, CTFreak is quite conservative in terms of memory consumption, so optimization tips will focus on CPU & disk space usage and I/O operations.

Disable project and task execution counters

By default, the project and task lists displayed in the web interface (and returned by the API) are accompanied by execution counters.

This is valuable information, but it can be costly to retrieve when you want to keep a large execution history.

If you notice a slowdown in consulting project and task lists, go to Settings → Global Settings menu and deactivate the retrieval of these counters via the dedicated checkbox.

Reduce the execution retention period

Experience has shown that when the execution history exceeds 8 million (depending on the characteristics of the host server), this can have a significative impact on CTFreak’s performance in terms of task scheduling accuracy and CPU usage.

As a reminder, execution retention period can be set globally, but also by project or even by task.

Don’t hesitate to use it to reduce the load on CTFreak, especially for high-frequency tasks that don’t necessarily need to keep a large execution history.

Reduce disk space used by execution logs

Most task types create one or more associated log/data files for each execution.

You can significantly reduce the disk space occupied by these files by keeping them only for failed or aborted executions, or even not keeping them at all if only the execution status matters to you.

Using an NVME drive for Config folder

CTFreak uses its configuration folder to store both its internal database and execution logs.

The installation prerequisites already specify that this folder should not be stored on a network drive (for performance reasons, but also to manage locks on access to the internal database).

However, you can go one step further and store this folder on an NVME drive to prevent I/O saturation in the event of high execution frequency.

Special case of high task execution frequency on the same server

Although CTFreak was originally designed for remote task execution on multiple servers, if you need to execute more than 40,000 command or script tasks per day on a single server, it’s better to install an instance of CTFreak on that server and use local command tasks (rather than launching remote executions).

Indeed, in the case of remote execution, the number of SSH connections that the target server has to handle, combined with the network connection, may make it impossible to maintain this execution frequency.

Do without powershell

When you run a local command task under Windows, the command line is by default launched via a powershell session.

In the case of intensive execution of this type of task, the instantiation of multiple powershell sessions consumes a lot of memory & CPU. It is therefore recommended to do without powershell via the “No Shell” option.