Server interaction

This section summarizes how to interact with the server. It serves two purposes:

  • a cheatsheet like refresher of the functionalities and their usage;
  • a detailed description of the functionalities.

Cheatsheet

Server domain:

https://tno-black-box-challenge-api.herokuapp.com
Table 1 Action cheatsheet
Action How Details
register via email, provide your name and affiliation Participation
evaluate evaluate(...).py/m Evaluate
submit submit(...).py/m Submit
show budget /show-my-budget?user_id=<string> Show budget
show submissions /show-my-submissions?user_id=<string>&set_id=<integer>&problem_id=<integer> Show submissions
Table 2 Language requirements
Language Version Remark
Python 2.x or 3.x utility functions provided
Matlab R2015a or above utility functions provided
Other language   user has to make utility functions

Details

Evaluate

Python

evaluate(username, password, set_id, problem_id, x)[source]

Evaluate a performance function

Parameters:
  • username (str) – Registered username for authentication. For testing without registration use ‘testuser’.
  • password (str) – Registered password for authentication. For testing without registration use ‘testpass’.
  • set_id (int) – Identification number of the problem set.
  • problem_id (int) – Identification number of the problem.
  • x (list, numpy.array) – Values of independent variables/random variables where the performance function is evaluated. Columns are the values of random variables (x1, x2,…xn). Bundle (vectorized) call is possible by providing multiple rows, each corresponds to one set of values of the random variables.
Returns:

  • g_val_sys (list (numpy.array)) – Performance function value on system level.
  • g_val_comp (list (numpy.array)) – Performance function value for each component.
  • msg (str) – Diagnostic message.

Examples

>>> g_val_sys, g_val_comp, msg = evaluate(username='testuser', password='testpass', set_id=-1, problem_id=2, x=[0.545, 1.23])
>>> print(g_val_sys)
1.2918

Matlab

See evaluate.m and parse_json.m on GitLab.

HTTP POST request

https://tno-black-box-challenge-api.herokuapp.com/evaluate
Table 3 Evaluate, input JSON body
Key Value Value example
username <string with quotes> ‘testuser’
password <string with quotes> ‘testpass’
set_id <integer> -1
problem_id <integer> 2
input_list <numbers in square bracket(s)> [0.545, 1.23] and [[0.545, 1.23], [0.6, 1.1]]

input_list contains values of independent variables/random variables where the performance function is evaluated. Columns are the values of random variables (x1, x2,…xn). Bundle (vectorized) call is possible by providing multiple rows, each corresponds to one set of values of the random variables.

{
  "username": "testuser",
  "password": "testpass",
  "set_ID": -1,
  "problem_ID": 2,
  "input_list": [0.545, 1.23]
}

The request returns the following JSON structure:

Table 4 Evaluate, output JSON body
Key Value Value example
g_val_sys <snumbers in square bracket(s)> [2.32]
g_val_comp <snumbers in square bracket(s)> [2.32, 5.76]
msg <string with quotes> ‘Ok’
{
  "g_val_sys": [2.32],
  "g_val_comp": [2.32, 5.76],
  "msg": "Ok"
}

Submit

Python

submit(username, password, set_id, problem_id, beta_sys, beta_comp=None, alpha_sys=None, alpha_comp=None)[source]

Evaluate a performance function

Parameters:
  • username (str) – Registered username for authentication. For testing without registration use ‘testuser’.
  • password (str) – Registered password for authentication. For testing without registration use ‘testpass’.
  • set_id (int) – Identification number of the problem set.
  • problem_id (int) – Identification number of the problem.
  • beta_sys (float) – System reliability index.
  • beta_comp (list of float) – Vector of reliability indices. Leave it empty [] if you do not want to submit it.
  • alpha_sys (list of float) – Vector of sensitivity factors. Leave it empty [] if you do not want to submit it.
  • alpha_comp (list of (list of) float) – Matrix of sensitivity factors. Each row is a vector component sensitivity factors. Leave it empty [] if you do not want to submit it.
Returns:

msg – Accompanying diagnostic message, e.g. warning.

Return type:

str

Examples

>>> msg = submit(username='testuser', password='testpass', set_id=-1, problem_id=3, beta_sys=3.4, beta_comp=np.array([3.4, 4.5]), alpha_sys=None, alpha_comp=np.array([[0.64, 0.77], [0.84, 0.54]]))
>>> print(msg)

Matlab

See submit.m on GitLab.

HTTP POST request

https://tno-black-box-challenge-api.herokuapp.com/submit
Table 5 Submit, input JSON body
Key Value Value example
username <string with quotes> ‘testuser’
password <string with quotes> ‘testpass’
set_id <integer> -1
beta_sys <number> 3.4
beta_comp <numbers in square bracket(s)> [3.4, 4.5]
alpha_sys <numbers in square bracket(s)> []
alpha_comp <numbers in square bracket(s)> [[0.64, 0.77], [0.84, 0.54]]
{
  "username": "testuser",
  "password": "testpass",
  "set_ID": -1,
  "problem_ID": 3,
  "beta_sys": 3.4,
  "beta_comp": [3.4, 4.5],
  "alpha_sys": [],
  "alpha_comp": [[0.64, 0.77], [0.84, 0.54]]
}

Show budget

HTTP GET request

Paste the URL below to the URL bar of your browser:

https://tno-black-box-challenge-api.herokuapp.com/show-my-budget?user_id=testid
Key Value Value example
user_id <string without quotes> testid

If you are registered to challenge you can use your user_id to check your consumed and available budget.


Show submissions

HTTP GET request

The server records all requests: x, g_val_sys, and g_val_comp values. This makes it possible to retain earlier evaluations in case your algorithm or computer crashed in the middle of the analysis and you lost the intermediate results.

Paste the URL below to the URL bar of your browser:

https://tno-black-box-challenge-api.herokuapp.com/show-my-submissions?user_id=testid&set_id=-1&problem_id=3
Key Value Value example
user_id <string without quotes> testid
set_id <integer> -1
problem_id <integer> 3

If you are registered to the challenge you can use your user_id to check your submissions.