Server interaction¶
This section summarizes how to interact with the server. It serves two purposes:
- a cheatsheet like refresher of the functionalities and their usage;
- a detailed description of the functionalities.
Cheatsheet¶
Server domain:
https://tno-black-box-challenge-api.herokuapp.com
Action | How | Details |
---|---|---|
register | via email, provide your name and affiliation | Participation |
evaluate |
evaluate(...).py/m |
Evaluate |
submit |
submit(...).py/m |
Submit |
show budget | /show-my-budget?user_id=<string> |
Show budget |
show submissions | /show-my-submissions?user_id=<string>&set_id=<integer>&problem_id=<integer> |
Show submissions |
Language | Version | Remark |
---|---|---|
Python | 2.x or 3.x | utility functions provided |
Matlab | R2015a or above | utility functions provided |
Other language | user has to make utility functions |
Details¶
Evaluate¶
Python¶
-
evaluate
(username, password, set_id, problem_id, x)[source]¶ Evaluate a performance function
Parameters: - username (str) – Registered username for authentication. For testing without registration use ‘testuser’.
- password (str) – Registered password for authentication. For testing without registration use ‘testpass’.
- set_id (int) – Identification number of the problem set.
- problem_id (int) – Identification number of the problem.
- x (list, numpy.array) – Values of independent variables/random variables where the performance function is evaluated. Columns are the values of random variables (x1, x2,…xn). Bundle (vectorized) call is possible by providing multiple rows, each corresponds to one set of values of the random variables.
Returns: - g_val_sys (list (numpy.array)) – Performance function value on system level.
- g_val_comp (list (numpy.array)) – Performance function value for each component.
- msg (str) – Diagnostic message.
Examples
>>> g_val_sys, g_val_comp, msg = evaluate(username='testuser', password='testpass', set_id=-1, problem_id=2, x=[0.545, 1.23]) >>> print(g_val_sys) 1.2918
Matlab¶
See evaluate.m and parse_json.m on GitLab.
HTTP POST request¶
https://tno-black-box-challenge-api.herokuapp.com/evaluate
Key | Value | Value example |
---|---|---|
username |
<string with quotes> | ‘testuser’ |
password |
<string with quotes> | ‘testpass’ |
set_id |
<integer> | -1 |
problem_id |
<integer> | 2 |
input_list |
<numbers in square bracket(s)> | [0.545, 1.23] and [[0.545, 1.23], [0.6, 1.1]] |
input_list
contains values of independent variables/random variables where the performance function is evaluated. Columns are the values of random variables (x1, x2,…xn). Bundle (vectorized) call is possible by providing multiple rows, each corresponds to one set of values of the random variables.
{
"username": "testuser",
"password": "testpass",
"set_ID": -1,
"problem_ID": 2,
"input_list": [0.545, 1.23]
}
The request returns the following JSON structure:
Key | Value | Value example |
---|---|---|
g_val_sys |
<snumbers in square bracket(s)> | [2.32] |
g_val_comp |
<snumbers in square bracket(s)> | [2.32, 5.76] |
msg |
<string with quotes> | ‘Ok’ |
{
"g_val_sys": [2.32],
"g_val_comp": [2.32, 5.76],
"msg": "Ok"
}
Submit¶
Python¶
-
submit
(username, password, set_id, problem_id, beta_sys, beta_comp=None, alpha_sys=None, alpha_comp=None)[source]¶ Evaluate a performance function
Parameters: - username (str) – Registered username for authentication. For testing without registration use ‘testuser’.
- password (str) – Registered password for authentication. For testing without registration use ‘testpass’.
- set_id (int) – Identification number of the problem set.
- problem_id (int) – Identification number of the problem.
- beta_sys (float) – System reliability index.
- beta_comp (list of float) – Vector of reliability indices. Leave it empty [] if you do not want to submit it.
- alpha_sys (list of float) – Vector of sensitivity factors. Leave it empty [] if you do not want to submit it.
- alpha_comp (list of (list of) float) – Matrix of sensitivity factors. Each row is a vector component sensitivity factors. Leave it empty [] if you do not want to submit it.
Returns: msg – Accompanying diagnostic message, e.g. warning.
Return type: str
Examples
>>> msg = submit(username='testuser', password='testpass', set_id=-1, problem_id=3, beta_sys=3.4, beta_comp=np.array([3.4, 4.5]), alpha_sys=None, alpha_comp=np.array([[0.64, 0.77], [0.84, 0.54]])) >>> print(msg)
HTTP POST request¶
https://tno-black-box-challenge-api.herokuapp.com/submit
Key | Value | Value example |
---|---|---|
username |
<string with quotes> | ‘testuser’ |
password |
<string with quotes> | ‘testpass’ |
set_id |
<integer> | -1 |
beta_sys |
<number> | 3.4 |
beta_comp |
<numbers in square bracket(s)> | [3.4, 4.5] |
alpha_sys |
<numbers in square bracket(s)> | [] |
alpha_comp |
<numbers in square bracket(s)> | [[0.64, 0.77], [0.84, 0.54]] |
{
"username": "testuser",
"password": "testpass",
"set_ID": -1,
"problem_ID": 3,
"beta_sys": 3.4,
"beta_comp": [3.4, 4.5],
"alpha_sys": [],
"alpha_comp": [[0.64, 0.77], [0.84, 0.54]]
}
Show budget¶
HTTP GET request¶
Paste the URL below to the URL bar of your browser:
https://tno-black-box-challenge-api.herokuapp.com/show-my-budget?user_id=testid
Key | Value | Value example |
---|---|---|
user_id |
<string without quotes> | testid |
If you are registered to challenge you can use your user_id
to check your consumed and available budget.
Show submissions¶
HTTP GET request¶
The server records all requests: x
, g_val_sys
, and g_val_comp
values. This makes it possible to retain earlier evaluations in case your algorithm or computer crashed in the middle of the analysis and you lost the intermediate results.
Paste the URL below to the URL bar of your browser:
https://tno-black-box-challenge-api.herokuapp.com/show-my-submissions?user_id=testid&set_id=-1&problem_id=3
Key | Value | Value example |
---|---|---|
user_id |
<string without quotes> | testid |
set_id |
<integer> | -1 |
problem_id |
<integer> | 3 |
If you are registered to the challenge you can use your user_id
to check your submissions.