tl;dr

Save the time of researching and evaluating optimization methods and, instead, focus on your work.

Cogneato is:

• Auto-configuring: Cogneato chooses the best method for your task, with support for up to 100 parameters, mixtures of parameter types, and noisy metric measurements.
• Always SOTA: We continuously review the latest research into experimental optimization methods, then incorporate selected methods into Cogneato's backend.

Cogneato Overview

Cogneato solves noisy, expensive-to-evaluate, black-box optimization problems. More succinctly, it does experimental optimization.

Cogneato has three components:

• Optimizer: A Bayesian optimizer.
• Router: A component that adapts the optimizer to your measurements.
• Interface: An interactive web tool that you can use any time without setup or configuration.

Optimizer

• Better than guessing: Finds parameter settings superior to those chosen manually, by experts
• Better than random search: Finds parameter settings using significantly fewer measurements than random search (which, in turn, outperforms grid search).
• Flexible: Since Bayesian optimization only uses metric measurements (no, ex., first or second derivatives), it can be applied to any parameterized system. The parameters you optimize may be real, ordinal, or categorical. Finally, you can optimize many different parameters simultaneously.

• Complexity: Bayesian optimization presents some mathematical and tooling overhead. One can't hack together a Bayesian optimization when-needed in the same way as one could quickly write a parameter sweep, grid search, or random search.
• Configurability: A Bayesian optimizer consists of several modules, each of which has several-to-many options for instantiation. You may choose a kernel, an acquisition function, parameter transformation methods, a trust region policy, and more. Configuring a Bayesian optimizer is a hard experimental optimization problem in it own right.
• Compute requirements: Traditionally, Bayesian optimization has been difficult to scale to large numbers of parameters or large numbers of measurements. However, recent developments have made solving larger-scale problems feasible.

By providing a Bayesian optimizer that's always at-the-ready, Cogneato removes the first disadvantage. You can get a high-quality parameter-set suggestion at any time from Cogneato without having to study, install, or code a Bayesian optimizer.

Cogneato combines high-quality open-source software, in-house implementations of new research, novel models (see Router, below), and extensive testing. Cogneato is tuned on a large suite of procedurally-generated functions and evaluated on a set of realistic test problems.

By keeping abreast of state-of-the-art methods and focusing our engineering efforts, we address the problem of high compute requirements.

Configurability is addressed by the router.

Router

After you submit your measurements table, the router featurizes it and maps the features to an appropriate configuration of a Bayesian optimizer. The configured optimizer is then used to generate an experiment design. This addresses the configurability disadvantage.

The router itself is parameterized. Its parameters are optimized by Cogneato.

Interface

When performance measurements take a long time -- hours or days -- it is common for an engineer to create a table, maintained by hand, of parameters and measurements. One records the parameters, waits for the measurement to complete, then enters the performance measurement value into the table. For this kind of workflow, there is little need to automate the optimization process, and, so, an interactive interface is preferred over an API. Spreadsheets and Jupyter notebooks are popular tools for collecting measurements.

Cogneato's interface was designed with this type of workflow in mind. You can copy a measurements table from a spreadsheet into the text box on the Cogneato home page, click Analyze, and you'll receive a new list of parameters -- an experiment design -- to evaluate.

Please send any questions or comments to help@cogneato.xyz.