Repository and Black-box reliability challenge 2019¶

This website contains a series of benchmark problems for testing reliability analysis methods. It consists of two main parts:

A repository for the Black-box Reliability Challenge 2019;
A reliability problem repository for benchmarking structural reliability method.

Note

The challenge is closed (see the Results) but you can still use the server. If you have any questions or suggestions feel free to contact us via email or Git Issues.

Introduction¶

What is this?¶

Black-box Reliability Challenge 2019 is a challenge and framework to test the performance of structural reliability methods. It is the first challenge in structural reliability where the reliability problems are truly black boxes for participants, meaning that their performance functions are hidden from the participants by making them accessible via the internet in the form of submitted input arguments and received output. To our knowledge, it is also the first on-line challenge in structural reliability. It is inspired by black-box optimization competitions such as BBComp. After the challenge, all the details of the reliability problems will be made publicly available, moving them to the repository for the structural reliability benchmarking framework.

We intend the current challenge to be the first iteration which we wish to improve based on the gained experience and your feedback. We are indebted to those who have already contributed to the design of the challenge and collecting reliability problems. We hope that the challenge and repository will evolve into an open, community project!

Motivation¶

Over the last decades, a plethora of reliability analysis methods has been developed with various capabilities, accuracy, and efficiency. Yet, an objective comparison and evaluation of these methods is missing. The currently available comparative studies are limited in scope and do not allow impartial analysis as only the final results are disclosed and they allow unrealistic exploration of the reliability problem by the analyst.

Here we propose a testing framework which is designed to:

deliver the long-needed, systematic and structured comparison of the performance of reliability algorithms among simplified, academic problems and real-life engineering problems;
provide invaluable guidance for practical applications of reliability analysis in the domains of geotechnical engineering, hydraulic engineering, structural engineering, and earthquake engineering; and
aid setting future research directions.

Timeline¶

Phase	Start date	End date
Workshop [Rozsas2018]	2018-Jan-24	NA
Announcement & test problems	2018-Dec-21	NA
Challenge	2019-March-28	2019-Dec-04
Announcement of results (Results)	2019-Dec-20	NA
Follow-up workshop	to be determined	NA

How to participate¶

Everyone is welcome to join the challenge (registration will open in early 2019). The participants use their computer and their selected method to solve reliability problems with performance functions located on the organizers servers. See the Quick start for a walk through how to do this.

Who we are¶

The idea of the Black-box Reliability Challenge, the benchmarking framework, and the reliability problem types are the outcome of a joint discussion and follow-up sessions between the participants of a workshop held in 2018 in Delft [Rozsas2018].

The organization and implementation of the challenge are spearheaded by the Department of Structural Reliability, TNO. The members of the team have selected the reliability problems which are known exclusively to them until the announcement of the results.

Results¶

Thank you everyone who participated in, contributed to, and/or was interested in the Black-box Reliability Challenge 2019. As a first step of disseminating the results we:

made all reference solutions and performance functions public, see Reliability problems;
summarize the main results in the plots below (participants are anonymized) (Visual summary).

If you use some code or results from this website please cite us (Citation).

If you have any suggestions and/or questions regarding the results or about the reliability challenge in general, do not hestitate to Contact us.

Visual summary¶

Note

To zoom into the images open them in a new tab or download them.

_images/beta_final_submission.facet.hidden_username.png

Comparison of submitted system reliability indices to reference solutions (horizonatal black lines).

_images/beta_vs_num_eval.facet.set1.hidden_username.png

Set 1. Submitted system reliability indices and corresponding number of performance function evaluations in comparison to reference solutions (horizonatal black lines).

_images/beta_vs_num_eval.facet.set2.hidden_username.png

Set 2. Submitted system reliability indices and corresponding number of performance function evaluations in comparison to reference solutions (horizonatal black lines).

Next steps¶

We are in a process of organizing a follow-up workshop that planned to take place in early 2020 in Delft, The Netherlands. We hope that the participants would share the details of their reliability algorithms and their experience for the benefit of the reliability community.

Participation¶

Everyone is welcome to join the challenge. There are two ways to access the reliability problems on the challenge server:

Without registration: using the testuser credentials given in Tutorial set. Without registration you have access only to the tutorial problems. To get full access you need to register, see the next point.
With registration: to get full access to the reliability problems and to participate in the challenge.

Registration¶

Note

The challenge is closed but you can still register to access the reliability problem!

You can register by sending an email to us expressing your intent. You will receive your username, password, and user_id which you can use to interact with the web server (see Quick start).

Please keep the following in mind:

In the email please indicate your name and affiliation (teams can also participate).
Use the following subject line in the email: Registration Black-box Reliability Challenge 2019.
One person is allowed to register only once (or part of only a single team).
Do not share your results and/or performance function evaluations with other participants.

We expect you to play fair and adhere to the single registration and not sharing principles. It is in our common interest to have everyone the same information available. After the completion of the challenge we will invite the participants to share their algorithms where others can verify the results.

Contribution¶

We would love to hear your opinion, suggestions, and have you contributing to this project. Please do not hesitate to contact us if you:

have any questions or issues;
have any suggestions;
would like to donate a reliability problem or two;
would like to contribute to the code.

You can get in touch via email or Git Issues. You can also directly contribute to the GitLab repository following the standard git procedure.

Quick start¶

This section walks you through the steps of evaluating performance functions and submitting results within the challenge framework.

Prerequisites:

internet connection;
basic Matlab or Python knowledge.

We provide a Python and a Matlab tutorial as well, pick the one you prefer.

Python¶

The most important actions you have to be familiar with to successfully participate in the challenge are:

Evaluate, and
Submit.

Evaluate evaluates a performance function on the web server, while Submit submits your intermediate and/or final results such as reliability indices and sensitivity factors to the server. Note that is of utmost importance that you also submit your intermediate results so we can compare the rate of convergence of different methods.

You need to have a Python installation (either 2.x or 3.x will work).

Evaluate¶

Download our evaluate.py file that manages the communication with the server and put into a folder of your choice.

Warning

GitLab will propose a filename that contains the path, rename it to evaluate.py.

Set the working directory of Python to this folder and open a new py-file (script) [*]. If you do not have the requests package installed (pip install requests) then install it as the Python files rely on it.

First, we need to provide our username and password. Once the challenge is open (Timeline) you can register and use your username and password to access the performance functions. For testing purposes let’s use a predefined, test user account:

username = 'testuser'
password = 'testpass'

Then we need some ids to uniquely identify our performance function. Let’s pick the second problem from the – Tutorial set.:

set_id = -1
problem_id  = 2

Problem 2 is corresponding to RP22 that is a two-dimensional, nonlinear function.

The next step is to specify the location where we would like to evaluate the performance function. The location is the independent variable of the function and we define it as a list with numerical items (you can also use a numpy array and bundle (vectorized) calls are also possible Evaluate):

x = [0.545, 1.23]

Now we have everything to evaluate the performance function. First we need to import the earlier downloaded evaluate.py function:

from evaluate import evaluate

Then we can use it to evaluate the performance function:

g_val_sys, g_val_comp, msg = evaluate(username, password, set_id, problem_id, x)
print(msg)
print(g_val_sys)
print(g_val_comp)

Where g_val_sys is the performance function value on system level and g_val_comp contains a performance function value for each component involved in the problem. For cases with a single performance function (as this one) g_val_sys=g_val_comp. Executing the above code prints the following to the command window:

Ok
1.2918
[1.2918]

Congratulations! You just evaluated a performance function on our server! Go ahead and test the function with other inputs, incorrect input, and try our other functions as well.

For using the performance function in your reliability algorithm you can wrap the above code into a function or you can create a lambda function that fits your reliability algorithm, e.g.

def g_fun(x):
        g = evaluate(username, password, set_id, problem_id, x)
        return g

Given that within this challenge we intentionally cap your number of performance function evaluations you might wonder how many evaluations you have left. You can check this by clicking on this link. The request_limit is set to -1 for this tutorial example which means that there is no upper limit so you can experiment with these functions. However, for the problems included in the challenge finite limits are enforced. Once you registered you will have your dedicated page to check your submissions.

[*]	You can download a file with all the code above from the GitLab repository: tutorial_evaluate.py.

Submit¶

After evaluating a performance function multiple times you might want to submit your intermediate or final results to the server to be compared with other participants. This section shows you how to do that. Note that you have to submit at least one result for a problem to be considered among the participants. Results submitted via other means than that described below are not accepted. You are strongly encouraged to submit your intermediate results as well since that will give us an idea about the rate of convergence of your method.

For submitting result you need to download the submit.py file. Save it into your working directory. For illustrative purposes let’s make a dummy a submission with the following input (you can use numpy arrays as well):

from submit import submit

username    = 'testuser'
password    = 'testpass'
set_id      = -1
problem_id  = 2
beta_sys    = 3.4
beta_comp   = 3.4
alpha_sys   = []
alpha_comp  = [0.64, 0.77]

You have to identify yourself and specify the particular set and problem you are submitting the results to. Note that some inputs are optional, e.g. alpha_sys is left empty in this case. The results can submitted by using the submit function:

msg = submit(username, password, set_id, problem_id, beta_sys, beta_comp, alpha_sys, alpha_comp)

print(msg)

To check your submission online, in the database of the web server click on this link.

Matlab¶

The most important actions you have to be familiar with to successfully participate in the challenge are:

Evaluate, and
Submit.

Evaluate evaluates a performance function on the web server, while Submit submits your intermediate and/or final results such as reliability indices and sensitivity factors to the server. Note that is of utmost importance that you also submit your intermediate results so we can compare the rate of convergence of different methods.

You need to have a Matlab installation (version R2015a or above).

Evaluate¶

Download our evaluate.m file that manages the communication with the server and parse_json.m that is used to post-process the response from the server. Put both files into a folder of your choice.

Warning

GitLab will propose a filename that contains the path, rename it to evaluate.m.

Set the working directory of Matlab to this folder and open a new m-file (script) [†].

First we need to provide our username and password. Once the challenge is open (Timeline) you can register and use your username and password to access the performance functions. For testing purposes let’s use a predefined, test user account:

username = 'testuser';
password = 'testpass';

Then we need some ids to uniquely identify our performance function. Let’s pick the second problem from the tutorial set (see – Tutorial set. for further details):

set_id = -1;
problem_id  = 2;

Problem 2 is corresponding to RP22 that is a two-dimensional, nonlinear function.

The next step is to specify the location where we would like to evaluate the performance function. The location is the independent variable of the function and we define it as a numerical vector (bundle (vectorized) calls are also possible by using matrices Evaluate):

x = [0.545, 1.23];

Now we have everything to evaluate the performance function for which we are going to use the earlier downloaded evaluate.m function:

[g_val_sys, g_val_comp, msg] = evaluate(username, password, set_id, problem_id, x);
disp(msg)
disp(g_val_sys)
disp(g_val_comp)

Where g_val_sys is the performance function value on system level and g_val_comp contains a performance function value for each component involved in the problem. For cases with a single performance function (as this one) g_val_sys=g_val_comp. Executing the above code prints the following to the command window:

Ok
 1.2918
 1.2918

Congratulations! You just evaluated a performance function on our server! Go ahead and test the function with other inputs, incorrect input, and try our other functions as well.

For using the performance function in your reliability algorithm you can wrap the above code into a function or you can create a anonymous function that fits your reliability algorithm, e.g.

g_fun = @(x) evaluate(username, password, set_id, problem_id, x);

Given that within this challenge we intentionally cap your number of performance function evaluations you might wonder how many evaluations you have left. You can check this by clicking on this link. The request_limit is set to -1 for this tutorial example which means that there is not upper limit so you can experiment with these functions. However, for the problems included in the challenge finite limits are enforced.

[†]	You can download a file with all the code above from the GitLab repository: tutorial_evaluate.m.

Submit¶

After evaluating a performance function multiple times you might want to submit your intermediate or final results to the server to be compared with other participants. This section shows you how to do that. Note that you have to submit at least one result for a problem to be considered among the participants. Results submitted via other means than that described below are not accepted. You are strongly encouraged to submit your intermediate results as well since that will give us an idea about the rate of convergence of your method.

For submitting result you need to download the submit.m file. Save it into your working directory. For illustrative purposes let’s make a dummy a submission with the following input:

username    = 'testuser';
password    = 'testpass';
set_id      = -1;
problem_id  = 2;
beta_sys    = 3.4;
beta_comp   = 3.4;
alpha_sys   = [];
alpha_comp  = [0.64, 0.77];

You have to identify yourself and specify the particular set and problem you are submitting the results to. Note that some inputs are optional, e.g. alpha_sys is left empty in this case. The results can submitted by using the submit function:

msg = submit(username, password, set_id, problem_id, beta_sys, beta_comp, alpha_sys, alpha_comp);

disp(msg)

To check your submission online, in the database of the web server click on this link.

Server interaction¶

This section summarizes how to interact with the server. It serves two purposes:

a cheatsheet like refresher of the functionalities and their usage;
a detailed description of the functionalities.

Cheatsheet¶

Server domain:

https://tno-black-box-challenge-api.herokuapp.com

Action cheatsheet¶
Action	How	Details
register	via email, provide your name and affiliation	Participation
`evaluate`	`evaluate(...).py/m`	Evaluate
`submit`	`submit(...).py/m`	Submit
show budget	`/show-my-budget?user_id=<string>`	Show budget
show submissions	`/show-my-submissions?user_id=<string>&set_id=<integer>&problem_id=<integer>`	Show submissions

Language requirements¶
Language	Version	Remark
Python	2.x or 3.x	utility functions provided
Matlab	R2015a or above	utility functions provided
Other language		user has to make utility functions

Details¶

Evaluate¶

Python¶

evaluate(username, password, set_id, problem_id, x)[source]¶

Evaluate a performance function

Parameters:

username (str) – Registered username for authentication. For testing without registration use ‘testuser’.
password (str) – Registered password for authentication. For testing without registration use ‘testpass’.
set_id (int) – Identification number of the problem set.
problem_id (int) – Identification number of the problem.
x (list, numpy.array) – Values of independent variables/random variables where the performance function is evaluated. Columns are the values of random variables (x1, x2,…xn). Bundle (vectorized) call is possible by providing multiple rows, each corresponds to one set of values of the random variables.

Returns:

g_val_sys (list (numpy.array)) – Performance function value on system level.
g_val_comp (list (numpy.array)) – Performance function value for each component.
msg (str) – Diagnostic message.

Examples

>>> g_val_sys, g_val_comp, msg = evaluate(username='testuser', password='testpass', set_id=-1, problem_id=2, x=[0.545, 1.23])
>>> print(g_val_sys)
1.2918

Matlab¶

See evaluate.m and parse_json.m on GitLab.

HTTP POST request¶

https://tno-black-box-challenge-api.herokuapp.com/evaluate

Evaluate, input JSON body¶
Key	Value	Value example
`username`	<string with quotes>	‘testuser’
`password`	<string with quotes>	‘testpass’
`set_id`	<integer>	-1
`problem_id`	<integer>	2
`input_list`	<numbers in square bracket(s)>	[0.545, 1.23] and [[0.545, 1.23], [0.6, 1.1]]

input_list contains values of independent variables/random variables where the performance function is evaluated. Columns are the values of random variables (x1, x2,…xn). Bundle (vectorized) call is possible by providing multiple rows, each corresponds to one set of values of the random variables.

{
  "username": "testuser",
  "password": "testpass",
  "set_ID": -1,
  "problem_ID": 2,
  "input_list": [0.545, 1.23]
}

The request returns the following JSON structure:

Evaluate, output JSON body¶
Key	Value	Value example
`g_val_sys`	<snumbers in square bracket(s)>	[2.32]
`g_val_comp`	<snumbers in square bracket(s)>	[2.32, 5.76]
`msg`	<string with quotes>	‘Ok’

{
  "g_val_sys": [2.32],
  "g_val_comp": [2.32, 5.76],
  "msg": "Ok"
}

Submit¶

Python¶

submit(username, password, set_id, problem_id, beta_sys, beta_comp=None, alpha_sys=None, alpha_comp=None)[source]¶

Evaluate a performance function

Parameters:	username (str) – Registered username for authentication. For testing without registration use ‘testuser’. password (str) – Registered password for authentication. For testing without registration use ‘testpass’. set_id (int) – Identification number of the problem set. problem_id (int) – Identification number of the problem. beta_sys (float) – System reliability index. beta_comp (list of float) – Vector of reliability indices. Leave it empty [] if you do not want to submit it. alpha_sys (list of float) – Vector of sensitivity factors. Leave it empty [] if you do not want to submit it. alpha_comp (list of (list of) float) – Matrix of sensitivity factors. Each row is a vector component sensitivity factors. Leave it empty [] if you do not want to submit it.
Returns:	msg – Accompanying diagnostic message, e.g. warning.
Return type:	str

Examples

>>> msg = submit(username='testuser', password='testpass', set_id=-1, problem_id=3, beta_sys=3.4, beta_comp=np.array([3.4, 4.5]), alpha_sys=None, alpha_comp=np.array([[0.64, 0.77], [0.84, 0.54]]))
>>> print(msg)

Matlab¶

See submit.m on GitLab.

HTTP POST request¶

https://tno-black-box-challenge-api.herokuapp.com/submit

Submit, input JSON body¶
Key	Value	Value example
`username`	<string with quotes>	‘testuser’
`password`	<string with quotes>	‘testpass’
`set_id`	<integer>	-1
`beta_sys`	<number>	3.4
`beta_comp`	<numbers in square bracket(s)>	[3.4, 4.5]
`alpha_sys`	<numbers in square bracket(s)>	[]
`alpha_comp`	<numbers in square bracket(s)>	[[0.64, 0.77], [0.84, 0.54]]

{
  "username": "testuser",
  "password": "testpass",
  "set_ID": -1,
  "problem_ID": 3,
  "beta_sys": 3.4,
  "beta_comp": [3.4, 4.5],
  "alpha_sys": [],
  "alpha_comp": [[0.64, 0.77], [0.84, 0.54]]
}

Show budget¶

HTTP GET request¶

Paste the URL below to the URL bar of your browser:

https://tno-black-box-challenge-api.herokuapp.com/show-my-budget?user_id=testid

Key	Value	Value example
`user_id`	<string without quotes>	testid

If you are registered to challenge you can use your user_id to check your consumed and available budget.

Show submissions¶

HTTP GET request¶

The server records all requests: x, g_val_sys, and g_val_comp values. This makes it possible to retain earlier evaluations in case your algorithm or computer crashed in the middle of the analysis and you lost the intermediate results.

Paste the URL below to the URL bar of your browser:

https://tno-black-box-challenge-api.herokuapp.com/show-my-submissions?user_id=testid&set_id=-1&problem_id=3

Key	Value	Value example
`user_id`	<string without quotes>	testid
`set_id`	<integer>	-1
`problem_id`	<integer>	3

If you are registered to the challenge you can use your user_id to check your submissions.

Challenge problems¶

Overview¶

The reliability problems – grouped into different sets – are summarized in the tables below. The sets are established to represent distinctively different problems where different methods are expected to excel, e.g. reliability problems with a single design point, or reliability problems with continuous and differentiable performance function. Sets and problems are identified with their set_id and problem_id, respectively; both are integers.

You can download all of the probabilistic models from the GitLab repository in a tabular format by clicking on this link.

Problem sets¶

Tutorial set¶

To help the understanding and verification purposes all details are disclosed for the first two problems in this set. Problem 3 exemplifies what information will be provided during the challenge for each reliability problems. Since these problems serve testing, no upper limit on the number of performance function evaluations is enforced. Use username: 'testuser' and password: 'testpass' for testing (only available for the tutorial).

`username`	‘testuser’
`password`	‘testpass’

– Tutorial set.¶
`set_id`	`problem_id`	Reliability problem	Max evaluations
-1	1	RP8	$\infty$
-1	2	RP22	$\infty$
-1	3	RP33	$\infty$

Challenge set 1 - single performance function¶

Reliability problems in this set have a single performance function but nothing else is disclosed about them. Once the challenge is open (Timeline) you can register and use your username and password to access these problems.

Evaluating the performance function yields a component and a system performance function value. These are equal as we have only a single performance function.

Note

Some of the performance functions rely on iterative numerical algorithms which might fail to converge under certain inputs; similarly as a finite element analysis. In those cases NaN values are returned. Make sure that your reliability algorithm can handle these cases. The performance functions will not yield NaN values for reasonable input points that do not correspond to failure.

– Challenge set 1.¶
`set_id`	`problem_id`	Reliability problem	Max evaluations
1	1	RP14	10000
1	2	RP24	10000
1	3	RP28	10000
1	4	RP31	10000
1	5	RP38	10000
1	6	RP53	10000
1	7	RP54	10000
1	8	RP63	10000
1	9	RP75	10000
1	10	RP107	10000
1	11	RP111	10000
1	12	RP201	10000
1	13	RP203	10000
1	14	RP213	10000
1	15	RP300	10000
1	16	RP301	10000

Challenge set 2 - Multiple, known number of performance functions¶

Reliability problems in this set have unknown number of performance functions. Once the challenge is open (Timeline) you can register and use your username and password to access these problems.

Evaluating the performance function yields performance function values for each component and the system as well. However, how the system performance function is calculated is not revealed, you test the RP33 system problem from the tutorial set.

Note

Some of the performance functions rely on iterative numerical algorithms which might fail to converge under certain inputs; similarly as a finite element analysis. In those cases NaN values are returned. Make sure that your reliability algorithm can handle these cases.

– Challenge set 2.¶
`set_id`	`problem_id`	Reliability problem	Max evaluations
2	1	RP25	10000
2	2	RP33	10000
2	3	RP35	10000
2	4	RP55	10000
2	5	RP57	10000
2	6	RP60	10000
2	7	RP77	10000
2	8	RP89	10000
2	9	RP91	10000
2	10	RP110	10000
2	11	RP202	10000

Challenge details¶

Accessing performance functions¶

The performance functions are located on a cloud server managed by the organizers. The functions are evaluated upon a request by the participants (Fig. 4). The evaluation request (interfacing with the server) is a HTTP (Hypertext Transfer Protocol) request that is the backbone of modern web development and ubiquitous in distributed system communication [Podila2013].

If you are using Python or Matlab you do not have to know anything about HTTP request as we created an easy-to-use function in each of these languages to access the performance functions on our server (see Evaluate). If you are using another language then you have to have a basic familiarity with HTTP, see Other languages.

High level overview of the challenge framework.

Other languages¶

If you are using a language other than Python or Matlab, or you want to make your own request directly you have to make a HTTP POST request with a JSON body to our server. The URL for the POST request: https://tno-black-box-challenge-api.herokuapp.com/evaluate. It should contain a JSON body with the following fields (the values are taken from Quick start):

{
   "username":'testuser',
   "password":'testpass',
   "set_ID": -1,
   "problem_ID" : 2,
   "input_list" : [0.545, 1.23]
}

The server returns a JSON file with 'msg', 'g_val_sys', and 'g_val_comp' fields.

The web server¶

The web server is hosted in the cloud by the challenge organizers and can be accessed via HTTP request. It is a Flask server, further description and a link to the code repository will come later.

Frequently asked questions¶

Can I submit multiple evaluation requests simultaneously?

Yes. Bundle (vectorized) call is possible by providing multiple rows in input_list, each corresponds to one set of values of the random variables. Columns are the values of random variables ($x_1, x_2,\cdots x_n$). For example making three evaluation requests of a two-dimensional problem:

in Python: input_list = [[0.545, 1.23], [0.6, 1.1], [0.4, 1.6]] or numpy.array([[0.545, 1.23], [0.6, 1.1], [0.4, 1.6]])
in Matlab: input_list = [0.545, 1.23; 0.6, 1.1; 0.4, 1.6]

What if I’m using a programming language other than Python or Matlab?

The challenge framework is designed in a way that you can use any programming language. If you are not using Python or Matlab then you have to make the HTTP request to our server yourself, see Server interaction.

Is the web server down? I receive an error message that my request is timed out.

You can check whether the web server is up by clicking on this link. You should see a plain welcome screen. If the server is down please contact us. If the server is up please contact us with the detailed description of your problem, and the programming language and operating system you are using.

My request has timed out. What can/should I do?

If the server is up, you can check it by clicking on this link, then the timeout might be caused by a large number of evaluations submitted at once (in a vectorized format). Since some performance functions involve (relatively) computationally demanding numerical algorithms, it can take a while to complete all the evaluations if you submit hundreds or thousands of them at once. You can either reduce the number of requested one-time evaluations and have multiple requests, or you can increase the waiting time of your local program which makes the request:

in Python: Open evaluate.py and change the number (30 seconds) in line r = requests.post(main_url + 'evaluate', json=body, timeout=30) to a greater one.
in Matlab: Open evaluate.m and change the number (30 seconds) in line options.Timeout = 30; to a greater one.

What is the airspeed velocity of an unladen swallow?

“Although a definitive answer would of course require further measurements, published species-wide averages of wing length and body mass, initial Strouhal estimates based on those averages and cross-species comparisons, the Lund wind tunnel study of birds flying at a range of speeds, and revised Strouhal numbers based on that study all lead me to estimate that the average cruising airspeed velocity of an unladen European Swallow is roughly 11 meters per second, or 24 miles an hour.” [Corum2003]

What is g_val_sys and g_val_comp and why are they equal?

g_val_sys is the performance function value on system level and g_val_comp contains a performance function value for each component involved in the problem. For cases with a single performance function value the component performance function is identical with the system performance function, g_val_sys=g_val_comp.

What is alpha_sys and alpha_comp?

They are sensitivity values/factors: direction cosines of the vector pointing from the origin to the design point in U-space. These sensitivity factors are often reported when FORM analysis performed and popular in standardization. For more details see section 8.1 in [Ditlevsen2007], where also alpha is used to denote the same variable (sensitivity factor).

What is the user_id for?

It can be used to:

Check your remaining number of performance function evaluations (Show budget), e.g. budget example.
Check your submissions (Show submissions), e.g. submission example.

Change the user_id in the URLs to yours to see your budget and submissions.

Can I contribute a reliability problem?

Sure! We would be happy to have your contribution. You can send it via email or add it the GitLab code repository directly. After the challenge has been completed the code of the web server and all the user reliability problems will be made available in a public repository.

Can I have access/download the probabilistic input for all reliability problems?

Yes. You can download the probabilistic models from the GitLab repository in a tabular format by clicking on this link.

Reliability problems¶

The structural reliability benchmarking framework is an on-line documentation platform that contains all kind of reliability problems with their descriptions, implementation files, and visualization of the performance functions. By making this information publicly available, it is intended to stimulate a systematic and reproducible assessment of the accuracy and efficiency of currently available reliability methods and those to come in the future. Furthermore, the platform is envisaged as a dynamic environment in which you are encouraged to contribute and add new, interesting problems.

Note

For most problems the failure probabilities are calculated with numerical methods that converge to the exact solution. Due to the applied stopping criteria of these numerical methods you may attain slightly different results. The difference should be a small difference in the second decimal of the normal form (scientific notation) representation of the failure probability. The reliability index ($\beta$) is calculated from the failure probability ($P_\mathrm{f}$) the following way: $\beta=-\Phi^{-1}(P_\mathrm{f})$.

RP8¶

Six-dimensional hyperplane.

– Tutorial set.¶
`set_id`	`problem_id`
-1	1

Overview¶

Category	Value
Type	symbolic
Number of random variables	6
Failure probability, $P_\mathrm{f}$	$7.84\cdot10^{-4}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	3.16
Number of performance functions	1
Reference	[Xu2018]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp}}}({\bf X}) = X_1 + 2X_2 + 2X_3 + X_4 - 5X_5 - 5X_6 \cr & {g_{\mathrm{sys}}}({\bf X}) = g_{\mathrm{comp}}({\bf X}) \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	Mean	Std
$X_1$	NA	Lognormal	4.783	0.09975	120.0	12.0
$X_2$	NA	Lognormal	4.783	0.09975	120.0	12.0
$X_3$	NA	Lognormal	4.783	0.09975	120.0	12.0
$X_4$	NA	Lognormal	4.783	0.09975	120.0	12.0
$X_5$	NA	Lognormal	3.892	0.198	50.0	10.0
$X_6$	NA	Lognormal	3.669	0.198	40.0	8.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_8(x)[source]¶

Performance function for reliability problem 8.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

Matlab¶

See gfun_8.m on GitLab.

RP14¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	1

Overview¶

Category	Value
Type	Symbolic
Number of random variables	5
Failure probability, $P_\mathrm{f}$	$7.52\cdot10^{-3}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	2.42
Number of performance functions	1
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp}}}({\bf X}) = {X_1} - {{32} \over {\pi X_2^3}} \cdot \root 2 \of {{{X_3^2X_4^2} \over {16}} + X_5^2} \cr & {g_{\mathrm{sys}}}({\bf X}) = {g_{\mathrm{comp}}}({\bf X}) \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	Mean	Std
$X_1$	NA	Uniform	70.0	80.0	75.0	2.887
$X_2$	NA	Normal	39.0	0.1	39.0	0.1
$X_3$	NA	Gumbel-max	1342.0	272.9	1500.0	350.0
$X_4$	NA	Normal	400.0	0.1	400.0	0.1
$X_5$	NA	Normal	250000.0	35000.0	250000.0	35000.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_14(x)[source]¶

Performance function for reliability problem 14.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP22¶

Quadratic function with mixed term, convex.

– Tutorial set.¶
`set_id`	`problem_id`
-1	2

Overview¶

Category	Value
Type	symbolic
Number of random variables	2
Failure probability, $P_\mathrm{f}$	$4.16\cdot10^{-3}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	2.64
Number of performance functions	1
Reference	[Grooteman2011]

Performance function¶

()¶\[ \eqalign{ & {g({\bf X})} = 2.5-\frac{(X_1 + X_2)}{\sqrt{2}} + 0.1\cdot (X_1 - X_2)^2 \cr & {g_{\mathrm{sys}}}({\bf X}) = g_{\mathrm{comp}}({\bf X}) \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_1$	NA	Normal	0.0	1.0			0.0	1.0
$X_2$	NA	Normal	0.0	1.0			0.0	1.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_22(x)[source]¶

Performance function for reliability problem 22.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

Matlab¶

See gfun_22.m on GitLab.

RP24¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	2

Overview¶

Category	Value
Type	Symbolic
Number of random variables	2
Failure probability, $P_\mathrm{f}$	$2.86\cdot10^{-3}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	2.76
Number of performance functions	1
Reference	[Dai2016]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp}}}({\bf X}) = 2.5 - 0.2357 \cdot ({X_1} - {X_2}) + 0.00463 \cdot {({X_1} + {X_2} - 20)^4} \cr & {g_{\mathrm{sys}}}({\bf X}) = {g_{\mathrm{comp}}}({\bf X}) \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_1$	NA	Normal	10.0	3.0			10.0	3.0
$X_2$	NA	Normal	10.0	3.0			10.0	3.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_24(x)[source]¶

Performance function for reliability problem 22.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP25¶

– Challenge set 2.¶
`set_id`	`problem_id`
2	1

Overview¶

Category	Value
Type	Symbolic
Number of random variables	2
Failure probability, $P_\mathrm{f}$	$6.14\cdot10^{-6}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	4.36
Number of performance functions	2
Reference	[Dai2016]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp},1}}({\bf X}) = X_1^2 - 8 \cdot {X_2} + 16 \cr & {g_{\mathrm{comp},2}}({\bf X}) = - 16{X_1} + {X_2} + 32 \cr & {g_{\mathrm{sys}}}({\bf X}) = \max \left\{ \matrix{ {g_{\mathrm{comp},1}}{\bf X}) \hfill \cr {g_{\mathrm{comp},2}}({\bf X}) \hfill \cr} \right. \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_{1}$	NA	Normal	0.0	1.0			0.0	1.0
$X_{2}$	NA	Normal	0.0	1.0			0.0	1.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_111(x)[source]¶

Performance function for reliability problem 111.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP28¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	3

Overview¶

Category	Value
Type	Symbolic
Number of random variables	2
Failure probability, $P_\mathrm{f}$	$1.46\cdot10^{-7}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	5.11
Number of performance functions	1
Reference	[Dai2016]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp}}}({\bf X}) = {X_1} \cdot {X_2} - 146.14 \cr & {g_{\mathrm{sys}}}({\bf X}) = {g_{\mathrm{comp}}}({\bf X}) \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_1$	NA	Normal	78064.0	11710.0			78064.0	11710.0
$X_2$	NA	Normal	0.0104	0.00156			0.0104	0.00156

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_28(x)[source]¶

Performance function for reliability problem 28.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP31¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	4

Overview¶

Category	Value
Type	Symbolic
Number of random variables	2
Failure probability, $P_\mathrm{f}$	$1.80\cdot10^{-4}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	3.58
Number of performance functions	1
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp}}}({\bf X}) = 2 - {X_2} + {(4 \cdot {X_1})^4} \cr & {g_{\mathrm{sys}}}({\bf X}) = {g_{\mathrm{comp}}}({\bf X}) \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_1$	NA	Normal	0.0	1.0			0.0	1.0
$X_2$	NA	Normal	0.0	1.0			0.0	1.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_31(x)[source]¶

Performance function for reliability problem 31.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP33¶

– Tutorial set.; – Challenge set 2.¶
`set_id`	`problem_id`
-1	3
2	2

Overview¶

Category	Value
Type	Symbolic
Number of random variables	3
Failure probability, $P_\mathrm{f}$	$2.57\cdot10^{-3}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	2.80
Number of performance functions	2
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp},1}}({\bf X}) = - {X_1} - {X_2} - {X_3} + 3 \cdot \root 2 \of 3 \cr & {g_{\mathrm{comp},2}}\left( {\bf X} \right) = - {X_3} + 3 \cr & {g_{\mathrm{sys}}}({\bf X}) = \min \left\{ \matrix{ {g_{\mathrm{comp},1}}({\bf X}) \hfill \cr {g_{\mathrm{comp},2}}\left( {\bf X} \right) \hfill \cr} \right. \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_2$	Std
$X_{1}$	NA	Normal	1.0	1.0
$X_{2}$	NA	Normal	1.0	1.0
$X_{3}$	NA	Normal	1.0	1.0

The random variables are mutually independent.

Visualization¶

Implementation¶

gfun_33(x)[source]¶

Performance function for reliability problem 33.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP35¶

– Challenge set 2.¶
`set_id`	`problem_id`
2	3

Overview¶

Category	Value
Type	Symbolic
Number of random variables	2
Failure probability, $P_\mathrm{f}$	$3.54\cdot10^{-3}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	2.70
Number of performance functions	2
Reference	[Dai2016]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp},1}}({\bf X}) = 2 - {X_2} + \exp ( - 0.1 \cdot X_1^2) + {(0.2 \cdot {X_1})^4} \cr & {g_{\mathrm{comp},2}}({\bf X}) = 4.5 - {X_1} \cdot {X_2} \cr & {g_{\mathrm{sys}}}({\bf X}) = \min \left\{ \matrix{ {g_{\mathrm{comp},1}}({\bf X}) \hfill \cr {g_{\mathrm{comp},2}}({\bf X}) \hfill \cr} \right. \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_{1}$	NA	Normal	0.0	1.0			0.0	1.0
$X_{2}$	NA	Normal	0.0	1.0			0.0	1.0

The random variables are mutually independent.

Visualization¶

Implementation¶

gfun_35(x)[source]¶

Performance function for reliability problem 35.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP38¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	5

Overview¶

Category	Value
Type	Symbolic
Number of random variables	7
Failure probability, $P_\mathrm{f}$	$8.10\cdot10^{-3}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	2.48
Number of performance functions	1
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp}}}({\bf X}) = 15.59 \cdot {10^4} - {{{X_1} \cdot X_2^3} \over {2 \cdot X_2^3}} \cdot {{X_4^2 - 4 \cdot {X_5} \cdot {X_6} \cdot X_7^2 + {X_4} \cdot ({X_6} + 4 \cdot {X_5} + 2 \cdot {X_6} \cdot {X_7})} \over {{X_4} \cdot {X_5} \cdot ({X_4} + {X_6} + 2 \cdot {X_6} \cdot {X_7})}} \cr & {g_{\mathrm{sys}}}({\bf X}) = {g_{\mathrm{comp}}}({\bf X}) \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	Mean	Std
$X_1$	NA	Normal	350.0	35.0	350.0	35.0
$X_2$	NA	Normal	50.8	5.08	50.8	5.08
$X_3$	NA	Normal	3.81	0.381	3.81	0.381
$X_4$	NA	Normal	173.0	17.3	173.0	17.3
$X_5$	NA	Normal	9.38	0.938	9.38	0.938
$X_6$	NA	Normal	33.1	3.31	33.1	3.31
$X_7$	NA	Normal	0.036	0.0036	0.036	0.0036

The random variables are mutually independent.

Visualization¶

Implementation¶

gfun_38(x)[source]¶

Performance function for reliability problem 38.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP53¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	6

Overview¶

Category	Value
Type	Symbolic
Number of random variables	2
Failure probability, $P_\mathrm{f}$	$3.13\cdot10^{-2}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	1.86
Number of performance functions	1
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp}}}({\bf X}) = \sin \left( {{{5 \cdot {X_1}} \over 2}} \right) + 2 - {{(X_1^2 + 4) \cdot ({X_2} - 1)} \over {20}} \cr & {g_{\mathrm{sys}}}({\bf X}) = {g_{\mathrm{comp}}}({\bf X}) \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_1$	NA	Normal	1.5	1.0			1.5	1.0
$X_2$	NA	Normal	2.5	1.0			2.5	1.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_53(x)[source]¶

Performance function for reliability problem 53.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP54¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	7

Overview¶

Category	Value
Type	Symbolic
Number of random variables	20
Failure probability, $P_\mathrm{f}$	$9.98\cdot10^{-4}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	3.09
Number of performance functions	1
Reference	[Dai2016]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp}}}({\bf X}) = \sum\limits_{i = 1}^{20} {{X_i} - 8.951} \cr & {g_{\mathrm{sys}}}({\bf X}) = {g_{\mathrm{comp}}}({\bf X}) \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_1$	NA	Exponential	1.0				1.0	1.0
$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$
$X_{20}$	NA	Exponential	1.0				1.0	1.0

The random variables are identically distributed and mutually independent.

Visualization¶

Implementation¶

gfun_54(x)[source]¶

Performance function for reliability problem 54.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP55¶

– Challenge set 2.¶
`set_id`	`problem_id`
2	4

Overview¶

Category	Value
Type	Symbolic
Number of random variables	2
Failure probability, $P_\mathrm{f}$	$3.60\cdot10^{-1}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	-0.15
Number of performance functions	4
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp},1}}({\bf X}) = 0.2 + 0.6 \cdot {({X_1} - {X_2})^4} - {{{X_1} - {X_2}} \over {\root 2 \of 2 }} \cr & {g_{\mathrm{comp},2}}({\bf X}) = 0.2 + 0.6 \cdot {({X_1} - {X_2})^4} + {{{X_1} - {X_2}} \over {\root 2 \of 2 }} \cr & {g_{\mathrm{comp},3}}({\bf X}) = {X_1} - {X_2} + {5 \over {\root 2 \of 2 }} - 2.2 \cr & {g_{\mathrm{comp},4}}({\bf X}) = {X_2} - {X_1} + {5 \over {\root 2 \of 2 }} - 2.2 \cr & {g_{\mathrm{sys}}}({\bf X}) = \min \left\{ \matrix{ {g_{\mathrm{comp},1}}({\bf X}) \hfill \cr {g_{\mathrm{comp},2}}({\bf X}) \hfill \cr {g_{\mathrm{comp},3}}({\bf X}) \hfill \cr {g_{\mathrm{comp},4}}({\bf X}) \hfill \cr} \right. \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_{1}$	NA	Uniform	-1.0	1.0			0.0	0.5774
$X_{2}$	NA	Uniform	-1.0	1.0			0.0	0.5774

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_55(x)[source]¶

Performance function for reliability problem 55.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP57¶

– Challenge set 2.¶
`set_id`	`problem_id`
2	5

Overview¶

Category	Value
Type	Symbolic
Number of random variables	2
Failure probability, $P_\mathrm{f}$	$2.84\cdot10^{-2}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	1.91
Number of performance functions	3
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp},1}}({\bf X}) = - X_1^2 + X_2^3 + 3 \cr & {g_{\mathrm{comp},2}}({\bf X}) = 2 - {X_1} - 8 \cdot {X_2} \cr & {g_{\mathrm{comp},3}}({\bf X}) = {({X_1} + 3)^2} + {({X_2} + 3)^2} - 4 \cr & {g_{\mathrm{sys}}}({\bf X}) = \min \left\{ \matrix{ \max \left\{ \matrix{ {g_{\mathrm{comp},1}}({\bf X}) \hfill \cr {g_{\mathrm{comp},2}}({\bf X}) \hfill \cr} \right. \hfill \cr {g_{\mathrm{comp},3}}({\bf X}) \hfill \cr} \right. \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_{1}$	NA	Normal	0.0	1.0			0.0	1.0
$X_{2}$	NA	Normal	0.0	1.0			0.0	1.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_57(x)[source]¶

Performance function for reliability problem 57.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP60¶

– Challenge set 2.¶
`set_id`	`problem_id`
2	6

Overview¶

Category	Value
Type	Symbolic
Number of random variables	5
Failure probability, $P_\mathrm{f}$	$4.56\cdot10^{-2}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	1.70
Number of performance functions	7
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp},1}}({\bf X}) = {X_1} - {X_5} \cr & {g_{\mathrm{comp},2}}({\bf X}) = {X_2} - {{{X_5}} \over 2} \cr & {g_{\mathrm{comp},3}}({\bf X}) = {X_3} - {{{X_5}} \over 2} \cr & {g_{\mathrm{comp},4}}({\bf X}) = {X_4} - {{{X_5}} \over 2} \cr & {g_{\mathrm{comp},5}}({\bf X}) = {X_2} - {X_5} \cr & {g_{\mathrm{comp},6}}({\bf X}) = {X_3} - {X_5} \cr & {g_{\mathrm{comp},7}}({\bf X}) = {X_4} - {X_5} \cr & {g_{\mathrm{sys}}}({\bf X}) = \min \left\{ \matrix{ {g_{\mathrm{comp},1}}({\bf X}) \hfill \cr \max \left\{ \matrix{ \min \left\{ \matrix{ {g_{\mathrm{comp},2}}({\bf X}) \hfill \cr {g_{\mathrm{comp},3}}({\bf X}) \hfill \cr {g_{\mathrm{comp},4}}({\bf X}) \hfill \cr} \right. \hfill \cr \max \left\{ \matrix{ \min \left\{ \matrix{ {g_{\mathrm{comp},5}}({\bf X}) \hfill \cr {g_{\mathrm{comp},6}}({\bf X}) \hfill \cr} \right. \hfill \cr {g_{\mathrm{comp},7}}({\bf X}) \hfill \cr} \right. \hfill \cr} \right. \hfill \cr} \right. \cr} $\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	Mean	Std
$X_{1}$	NA	Lognormal	7.691	0.09975	2200.0	220.0
$X_{2}$	NA	Lognormal	7.645	0.09975	2100.0	210.0
$X_{3}$	NA	Lognormal	7.736	0.09975	2300.0	230.0
$X_{4}$	NA	Lognormal	7.596	0.09975	2000.0	200.0
$X_{5}$	NA	Lognormal	7.016	0.3853	1200.0	480.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_60(x)[source]¶

Performance function for reliability problem 60.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP63¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	8

Overview¶

Category	Value
Type	Symbolic
Number of random variables	100
Failure probability, $P_\mathrm{f}$	$3.79\cdot10^{-4}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	3.36
Number of performance functions	1
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp}}}({\bf X}) = 0.1 \cdot \sum\limits_{i = 2}^{100} {X_i^2} - {X_1} - 4.5 \cr & {g_{\mathrm{sys}}}({\bf X}) = {g_{\mathrm{comp}}}({\bf X}) \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_1$	NA	Normal	0.0	1.0			0.0	1.0
$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$
$X_{100}$	NA	Normal	0.0	1.0			0.0	1.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_63(x)[source]¶

Performance function for reliability problem 63.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP75¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	9

Overview¶

Category	Value
Type	Symbolic
Number of random variables	2
Failure probability, $P_\mathrm{f}$	$1.07\cdot10^{-2}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	2.33
Number of performance functions	1
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp}}}({\bf X}) = 3 - {X_1} \cdot {X_2} \cr & {g_{\mathrm{sys}}}({\bf X}) = {g_{\mathrm{comp}}}({\bf X}) \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_1$	NA	Normal	0.0	1.0			0.0	1.0
$X_2$	NA	Normal	0.0	1.0			0.0	1.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_75(x)[source]¶

Performance function for reliability problem 75.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP77¶

– Challenge set 2.¶
`set_id`	`problem_id`
2	7

Overview¶

Category	Value
Type	Symbolic
Number of random variables	3
Failure probability, $P_\mathrm{f}$	$2.87\cdot10^{-7}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	5.00
Number of performance functions	2
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp}}}({\bf X}) = \left\{ \matrix{ {X_1} - {X_2} - {X_3}{\rm{ , }}{X_3} \le 5 \hfill \cr {{\rm{X}}_3}{\rm{ - }}{{\rm{X}}_2}{\rm{ , }}{X_3} > 5{\rm{ }} \hfill \cr} \right. \cr & {g_{\mathrm{sys}}}({\bf X}) = {g_{\mathrm{comp}}}({\bf X}) \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	Mean	Std
$X_{1}$	NA	Normal	10.0	0.5	10.0	0.5
$X_{2}$	NA	Normal	0.0	1.0	0.0	1.0
$X_{3}$	NA	Normal	4.0	1.0	4.0	1.0

The random variables are mutually independent.

Visualization¶

Python¶

Implementation¶

gfun_77(x)[source]¶

Performance function for reliability problem 77.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP89¶

– Challenge set 2.¶
`set_id`	`problem_id`
2	8

Overview¶

Category	Value
Type	Symbolic
Number of random variables	2
Failure probability, $P_\mathrm{f}$	$5.43\cdot10^{-3}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	2.55
Number of performance functions	2
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp},1}}({\bf X}) = - (X_1^2 + {X_2} - 8) \cr & {g_{\mathrm{comp},2}}({\bf X}) = - \left( {{{{X_1}} \over 5} + {X_2} - 6} \right) \cr & {g_{\mathrm{sys}}}({\bf X}) = \min \left\{ \matrix{ {g_{\mathrm{comp},1}}({\bf X}) \hfill \cr {g_{\mathrm{comp},2}}({\bf X}) \hfill \cr} \right. \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_{1}$	NA	Normal	0.0	1.0			0.0	1.0
$X_{2}$	NA	Normal	0.0	1.0			0.0	1.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_89(x)[source]¶

Performance function for reliability problem 89.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP91¶

– Challenge set 2.¶
`set_id`	`problem_id`
2	9

Overview¶

Category	Value
Type	Symbolic
Number of random variables	5
Failure probability, $P_\mathrm{f}$	$6.97\cdot10^{-4}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	3.19
Number of performance functions	3
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp},1}}({\bf X}) = 0.847 + 0.96 \cdot {X_2}{\rm{ + }}0.986 \cdot {X_3} - 0.216 \cdot {X_4} + {\rm{ }}0.077 \cdot X_2^2 + 0.11 \cdot X_3^2 + {{0.007 \cdot X_{_4}^2} \over {0.378}} - {\rm{ }}{X_2} \cdot {\rm{ }}{X_3} - 0.106{\rm{ }} \cdot {X_2} \cdot {X_4} - {\rm{ }}0.11 \cdot {X_3} \cdot {X_4} \cr & {g_{\mathrm{comp},2}}({\bf X}) = {{8400 \cdot {X_1}} \over {\root 2 \of {X_3^2 + X_4^2 - {X_3} \cdot {X_4} + 3 \cdot X_5^2} }} - 1 \cr & {g_{\mathrm{comp},3}}({\bf X}) = {{8400 \cdot {X_1}} \over {\left| {{X_4}} \right|}} - 1 \cr & g({X}) = \min \left\{ \matrix{ {g_{\mathrm{comp},1}}({\bf X}) \hfill \cr {g_{\mathrm{comp},2}}({\bf X}) \hfill \cr {g_{\mathrm{comp},3}}({\bf X}) \hfill \cr} \right. \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	Mean	Std
$X_{1}$	NA	Normal	0.07433	0.005	0.07433	0.005
$X_{2}$	NA	Normal	0.1	0.01	0.1	0.01
$X_{3}$	NA	Normal	13.0	60.0	13.0	60.0
$X_{4}$	NA	Normal	4751.0	48.0	4751.0	48.0
$X_{5}$	NA	Normal	-684.0	11.0	-684.0	11.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_91(x)[source]¶

Performance function for reliability problem 91.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP107¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	10

Overview¶

Category	Value
Type	Symbolic
Number of random variables	10
Failure probability, $P_\mathrm{f}$	$2.92\cdot10^{-7}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	5.0
Number of performance functions	1
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp}}}({\bf X}) = 5 \cdot \root 2 \of {10} - \sum\limits_{i = 1}^{10} {{X_i}} \cr & {g_{\mathrm{sys}}}({\bf X}) = {g_{\mathrm{comp}}}({\bf X}) \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_1$	NA	Normal	0.0	1.0			0.0	1.0
$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$
$X_{10}$	NA	Normal	0.0	1.0			0.0	1.0

The random variables are identically distributed and mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_107(x)[source]¶

Performance function for reliability problem 107.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP110¶

– Challenge set 2.¶
`set_id`	`problem_id`
2	10

Overview¶

Category	Value
Type	Symbolic
Number of random variables	2
Failure probability, $P_\mathrm{f}$	$3.19\cdot10^{-5}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	4.00
Number of performance functions	2
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp},1}}({\bf X}) = \left\{ \matrix{ 4 - {X_1}{\rm{ , }}{X_1} > 3.5{\rm{ }} \hfill \cr 0.85 - 0.1 \cdot {X_1}{\rm{ , }}{X_1} \le 3.5 \hfill \cr} \right. \cr & {g_{\mathrm{comp},2}}({\bf X}) = \left\{ \matrix{ 0.5 - 0.1 \cdot {X_2}{\rm{ , }}{X_2} > 2 \hfill \cr 2.3 - {X_2}{\rm{ , }}{X_2} \le 2 \hfill \cr} \right. \cr & g({\bf X}) = \min \left\{ \matrix{ {g_{\mathrm{comp},1}}({\bf X}) \hfill \cr {g_{\mathrm{comp},2}}({\bf X}) \hfill \cr} \right. \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_{1}$	NA	Normal	0.0	1.0			0.0	1.0
$X_{2}$	NA	Normal	0.0	1.0			0.0	1.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_110(x)[source]¶

Performance function for reliability problem 25.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP111¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	11

Overview¶

Category	Value
Type	Symbolic
Number of random variables	2
Failure probability, $P_\mathrm{f}$	$7.65\cdot10^{-7}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	4.81
Number of performance functions	1
Reference	[Schueller2004]

Performance function¶

()¶\[ \eqalign{ & {g_{\mathrm{comp}}}({\bf X}) = {{{5^2}} \over 2} - \left| {{X_1} \cdot {X_2}} \right| \cr & {g_{\mathrm{sys}}}({\bf X}) = {g_{\mathrm{comp}}}({\bf X}) \cr}\]

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_1$	NA	Normal	0.0	1.0			0.0	1.0
$X_2$	NA	Normal	0.0	1.0			0.0	1.0

The random variables are mutually independent.

Visualization¶

Implementation¶

Python¶

gfun_111(x)[source]

Performance function for reliability problem 111.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP201¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	12

Overview¶

Category	Value
Type	Finite element
Number of random variables	21
Failure probability, $P_\mathrm{f}$	$1.05\cdot10^{-4}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	3.7
Number of performance functions	1
Reference	[Blatman2010]

Performance function¶

Too complex to be written here, see the implementation in the code repository.

Random variables¶

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_1$	NA	Normal	0.0	1.0			0.0	1.0
$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$
$X_{21}$	NA	Normal	0.0	1.0			0.0	1.0

The random variables are mutually independent.

Visualization¶

_static/gfun_matrix_plot/rp_201_matrix.png

Implementation¶

Python¶

gfun_201(x)[source]¶

Performance function for reliability problem 201.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP202¶

– Challenge set 2.¶
`set_id`	`problem_id`
2	11

Overview¶

Category	Value
Type	Finite element
Number of random variables	225
Failure probability, $P_\mathrm{f}$	$6.03\cdot10^{-5}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	3.43
Number of performance functions	2
Reference	[Schueller2007]

Performance function¶

Too complex to be written here, see the implementation in the code repository.

Random variables¶

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_1$	NA	Normal	0.0	1.0			0.0	1.0
$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$
$X_{225}$	NA	Normal	0.0	1.0			0.0	1.0

The random variables are mutually independent.

Visualization¶

_static/gfun_matrix_plot/rp_202_matrix.png

Implementation¶

Python¶

gfun_202(x)[source]¶

Performance function for reliability problem 202.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP203¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	13

Overview¶

Category	Value
Type	Finite element/ symbolic
Number of random variables	4
Failure probability, $P_\mathrm{f}$	$4.33\cdot10^{-7}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	4.92
Number of performance functions	1
Reference	DACC

Performance function¶

Too complex to be written here, see the implementation in the code repository.

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_2$	Std
$X_{1}$	NA	Normal	1.0	1.0
$X_{2}$	NA	Normal	1.0	1.0
$X_{3}$	NA	Normal	1.0	1.0
$X_{4}$	NA	Normal	1.0	1.0

The random variables are mutually independent.

Visualization¶

_static/gfun_matrix_plot/rp_203_matrix.png

Implementation¶

Python¶

gfun_203(x)[source]¶

Performance function for reliability problem 203. Bram’s geotechnical problem.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP213¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	14

Overview¶

Category	Value
Type	Finite element
Number of random variables	13
Failure probability, $P_\mathrm{f}$	$2.84\cdot10^{-4}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	3.45
Number of performance functions	1
Reference	DACC

Performance function¶

Too complex to be written here, see the implementation in the code repository.

Random variables¶

Variable	Description	Distribution	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	Mean	Std
$X_1$	NA	Normal	0.0	1.0			0.0	1.0
$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$
$X_{13}$	NA	Normal	0.0	1.0			0.0	1.0

The random variables are mutually independent.

Visualization¶

_static/gfun_matrix_plot/rp_213_matrix.png

Implementation¶

Python¶

gfun_213_le_frame(x)[source]¶

Performance function for reliability problem 213.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP300¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	15

Overview¶

Category	Value
Type	Finite element
Number of random variables	14
Failure probability, $P_\mathrm{f}$	$5.22\cdot10^{-5}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	3.88
Number of performance functions	1
Reference	[Sepulveda2019]

Performance function¶

Too complex to be written here, see the implementation in the code repository.

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$
$X_1$	DACC	Normal	0	1
$X_2$	DACC	Normal	0	1
$\vdots$	$\vdots$	$\vdots$	$\vdots$	$\vdots$
$X_{14}$	DACC	Normal	0	1

The random variables are mutually independent.

Visualization¶

_static/gfun_matrix_plot/rp_300_matrix.png

Implementation¶

Python¶

gfun_300(x)[source]¶

Performance function for reliability problem 300.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

RP301¶

– Challenge set 1.¶
`set_id`	`problem_id`
1	16

Overview¶

Category	Value
Type	Finite element
Number of random variables	12
Failure probability, $P_\mathrm{f}$	$6.9\cdot10^{-5}$
Reliability index, $\beta=-\Phi^{-1}(P_\mathrm{f})$	3.81
Number of performance functions	1
Reference	[Slobbe2019]

Performance function¶

Too complex to be written here, see the implementation in the code repository.

Random variables¶

The parametrization of distributions follows that of in Distributions.

Variable	Description	Distribution	$\theta_1$	$\theta_2$	Mean	Std
$X_{1}$	NA	Lognormal	4.476	0.05677	88.0	5.0
$X_{2}$	NA	Lognormal	6.084	0.0681	440.0	30.0
$X_{3}$	NA	Lognormal	6.18	0.0681	484.0	33.0
$X_{4}$	NA	Lognormal	-2.663	0.08982	0.07	0.0063
$X_{5}$	NA	Lognormal	6.378	0.06941	590.0	41.0
$X_{6}$	NA	Lognormal	6.473	0.06925	649.0	45.0
$X_{7}$	NA	Lognormal	-2.663	0.08982	0.07	0.0063
$X_{8}$	NA	Normal	590.0	59.0	590.0	59.0
$X_{9}$	NA	Gumbel-max	268.4	46.0	295.0	59.0
$X_{10}$	NA	Lognormal	0.03797	0.04997	1.04	0.052
$X_{11}$	NA	Lognormal	-0.004975	0.09975	1.0	0.1
$X_{12}$	NA	Lognormal	-0.004975	0.09975	1.0	0.1

The random variables are mutually independent.

Visualization¶

_static/gfun_matrix_plot/rp_301_matrix.png

Implementation¶

Python¶

gfun_301(x)[source]¶

Performance function for reliability problem 301. High strength RC deep beam.

Parameters:	x (numpy.array of float(s)) – Values of independent variables: columns are the different parameters/random variables (x1, x2,…xn) and rows are different parameter/random variables sets for different calls.
Returns:	g_val_sys (numpy.array of float(s)) – Performance function value for the system. g_val_comp (numpy.array of float(s)) – Performance function value for each component. msg (str) – Accompanying diagnostic message, e.g. warning.

Distributions¶

Continuous:
- Normal
- Lognormal
- Gumbel-max
- Gamma
- Uniform
- Truncated Normal
- Exponential

Normal¶

Parametrization¶

Category	Value
Type	univariate, continuous
Support	$x \in (-\infty,\infty)$
Parameter	$\theta_1 = \mu \in (-\infty,\infty)$, location
Parameter	$\theta_2 = \sigma \in (0,\infty)$, scale
Mean	$\mu$
Variance	$\sigma^2$

Probability density function¶

()¶\[ f(x \mid \mu ,\sigma) = {\frac {1}{\sqrt {2\pi \sigma ^{2}}}} \cdot e^{-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}} = \phi \left( {\frac{{x - \mu }}{\sigma }} \right)\]

Cumulative distribution function¶

()¶\[ F(x \mid \mu ,\sigma ) = \frac{1}{2} \cdot \left( {1 + erf\left( {\frac{{x - \mu }}{{\sigma \cdot \sqrt 2 }}} \right)} \right) = \Phi \left( {\frac{{x - \mu }}{\sigma }} \right)\]

Lognormal¶

Parametrization¶

Category	Value
Type	univariate, continuous
Support	$x \in (0,\infty)$
Parameter	$\theta_1 = \mu \in (-\infty,\infty)$, shape
Parameter	$\theta_2 = \sigma \in (0,\infty)$, scale
Mean	${e^{\left( {\mu + \frac{{{\sigma ^2}}}{2}} \right)}}$
Variance	$\left( {{e^{{\sigma ^2}}} - 1} \right) \cdot {e^{2\mu + {\sigma ^2}}}$

Probability density function¶

()¶\[ f(x \mid \mu ,\sigma) = {\frac {1}{x}}\cdot {\frac {1}{\sigma {\sqrt {2\pi \,}}}} \cdot \exp \left(-{\frac {(\ln x-\mu )^{2}}{2\sigma ^{2}}}\right)\]

Cumulative distribution function¶

()¶\[ F(x, \mid \mu ,\sigma ) = \frac{1}{2} \cdot \left( {1 + erf\left( {\frac{{\ln (x) - \mu }}{{\sigma \cdot \sqrt 2 }}} \right)} \right) = \Phi \left( {\frac{{\ln (x) - \mu }}{\sigma }} \right)\]

Gumbel-max¶

Parametrization¶

Category	Value
Type	univariate, continuous
Support	$x \in (-\infty,\infty)$
Parameter	$\theta_1 = \mu \in (-\infty,\infty)$, location
Parameter	$\theta_2 = \beta \in (0,\infty)$, scale
Mean	$\mu + \beta \cdot 0.5772$[*]
Variance	$\frac{{{\pi ^2}}}{6} \cdot {\beta ^2}$

[*]	Euler-Mascheroni constant.

Probability density function¶

()¶\[ f(x \mid \mu ,\beta ) = \frac{1}{\beta } \cdot \exp \left( { - \frac{{x - \mu }}{\beta } - \exp \left( { - \frac{{x - \mu }}{\beta }} \right)} \right)\]

Cumulative distribution function¶

()¶\[ F(x \mid \mu ,\beta ) = \exp \left( { - \exp \left( { - \frac{{x - \mu }}{\beta }} \right)} \right)\]

Gamma¶

Parametrization¶

Category	Value
Type	univariate, continuous
Support	$x \in (0,\infty )$
Parameter	$\theta_1 = k \in (0,\infty )$, shape
Parameter	$\theta_2 = \theta \in (0,\infty )$, scale
Mean	$k \cdot \theta$
Variance	$k \cdot \theta^2$

Probability density function¶

()¶\[ f(x \mid k,\theta ) = \frac{1}{{\Gamma(k) \cdot {\theta ^k}}} \cdot {x^{k - 1}} \cdot {e^{ - \frac{x}{\theta }}}\]

Where $\Gamma(.)$ is the gamma function.

Cumulative distribution function¶

()¶\[ F(x \mid k,\theta ) = \frac{1}{{\Gamma (k)}} \cdot \gamma \left( {k,\frac{x}{\theta }} \right)\]

Where $\gamma(.)$ is the lower incomplete gamma function.

Uniform¶

Parametrization¶

Category	Value
Type	univariate, continuous
Support	$x \in [a,b]$
Parameter	$\theta_1 = a \in (-\infty,b)$, lower bound
Parameter	$\theta_2 = b \in (a,\infty)$, upper bound
Mean	${\tfrac {1}{2}} \cdot (a+b)$
Variance	${\tfrac {1}{12}} \cdot (b-a)^{2}$

Probability density function¶

()¶\[\begin{split} f(x \mid a,b ) = {\begin{cases}{\frac {1}{b-a}}&{\text{for }}x\in [a,b]\\0&{\text{otherwise}}\end{cases}}\end{split}\]

Cumulative distribution function¶

()¶\[\begin{split} F(x \mid a,b ) = {\begin{cases}0&{\text{for }}x<a\\{\frac {x-a}{b-a}}&{\text{for }}x\in [a,b)\\1&{\text{for }}x\geq b\end{cases}}\end{split}\]

Truncated Normal¶

Parametrization¶

Category	Value
Type	univariate, continuous
Support	$x \in [a,b]$
Parameter	$\theta_1 = \mu \in (-\infty,\infty)$, location
	$\theta_2 = \sigma \in (0,\infty)$, scale [†]
	$\theta_3 = a \in (-\infty,b)$, lower bound
	$\theta_4 = b \in (a,\infty)$, upper bound
Mean	$\mu +{\frac {\phi (\alpha )-\phi (\beta )}{Z}} \cdot \sigma$
Variance	${\sigma ^2} \cdot \left[ {1 + \frac{{\alpha \cdot \phi (\alpha ) - \beta \cdot \phi (\beta )}}{Z} - {{\left( {\frac{{\phi (\alpha ) - \phi (\beta )}}{Z}} \right)}^2}} \right]$

[†]	For a more rigorous definition see the Wikipedia page of the truncated normal distribution.

Probability density function¶

()¶\[\begin{split} f(x \mid \mu ,\sigma ,a,b) = \left\{ {\begin{array}{*{20}{l}}{\frac{{\phi (\xi )}}{{\sigma \cdot Z}}{\mkern 1mu} }&{{\text{for }}x \in [a,b]}\\0&{{\rm{otherwise}}}\end{array}} \right.\end{split}\]

Where the auxiliary variables are defines as:

()¶\[\begin{split} \begin{array}{l}\xi = \frac{{x - \mu }}{\sigma },\;\alpha = \frac{{a - \mu }}{\sigma },\;\beta = \frac{{b - \mu}}{\sigma }\\Z = \Phi (\beta ) - \Phi (\alpha )\end{array}\end{split}\]

and $\phi(.)$ and $\Phi(.)$ are the standard normal probability density function (Eq. (24)) and cumulative distribution function (Eq. (25)), respectively.

Cumulative distribution function¶

()¶\[\begin{split} F(x\mid \mu ,\sigma ,a,b) = \left\{ {\begin{array}{*{20}{l}}0&{{\text{for }}x < a}\\{\frac{{\Phi (\xi ) - \Phi (\alpha )}}{Z}}&{{\text{for }}x \in [a,b)}\\1&{{\text{for }}x \ge b}\end{array}} \right.\end{split}\]

Exponential¶

Parametrization¶

Category	Value
Type	univariate, continuous
Support	$x \in (0,\infty )$
Parameter	$\theta_1 = \lambda \in (0,\infty )$, rate
Mean	$\lambda^{-1}$
Variance	$\lambda^{-2}$

Probability density function¶

()¶\[ f(x \mid \lambda ) = \lambda \cdot {e^{ - \lambda x}}\]

Cumulative distribution function¶

()¶\[ F(x \mid \lambda ) = 1-e^{ - \lambda x}\]

References¶

[Grooteman2011]

Grooteman, F. (2011). An adaptive directional importance sampling method for structural reliability. Probabilistic Engineering Mechanics, 26(2), 134-141. doi:http://dx.doi.org/10.1016/j.probengmech.2010.11.002.

[Xu2018]

Xu J., Kong F. (2018). A new unequal-weighted sampling method for efficient reliability analysis. Reliability Engineering & System Safety. Volume 172. 2018. Pages 94-102.

[Rozsas2018]

Rozsas A., Slobbe A., Meinen N. (2018). Computational Challenges in the Reliability Assessment of Engineering Structures. 2018 January 24. Delft. https://www.reliabilitytno.com/.

[Podila2013]

Podila P. (2013). HTTP: The Protocol Every Web Developer Must Know - Part 1. 2013 April 8. https://code.tutsplus.com/tutorials/http-the-protocol-every-web-developer-must-know-part-1–net-31177.

[Corum2003]

Corum J. (2003). Estimating the Airspeed Velocity of an Unladen Swallow. 2003 November 17. http://style.org/unladenswallow/.

[Ditlevsen2007]

Ditlevsen O., Madsen H.O. (2007). Structural Reliability Methods. June-September 2007. http://od-website.dk/books/OD-HOM-StrucRelMeth-Ed2.3.7.pdf.

[Schueller2004]

Schuëller, G.I. & Pradlwarter, Helmut & Koutsourelakis, P. (2004). A critical appraisal of reliability estimation procedures for high dimensions. Probabilistic Engineering Mechanics. 19. Pages 463-474. doi: https://doi.org/10.1016/j.probengmech.2004.05.004.

[Dai2016]

Dai, H., Zhang, H., & Wang, W. (2016). A new maximum entropy-based importance sampling for reliability analysis. doi: https://doi.org/10.1016/j.strusafe.2016.08.001.

[Sepulveda2019]

Sepulved, J., Faber, M. (2019). Benchmark of Emerging Structural Reliability Methods. doi: https://doi.org/10.22725/ICASP13.439.

[Slobbe2019]

Slobbe, A. Rozsas, A. Allaix, D. Bigaj-van Vliet, A. (2019). On the value of a reliability‐based nonlinear finite element analysis approach in the assessment of concrete structures. doi: https://doi.org/10.1002/suco.201800344.

[Schueller2007]

Schuëller, G.I. Pradlwarter, H.J. (2007). Benchmark study on reliability estimation in higher dimensions of structural systems – An overview. doi: https://doi.org/10.1016/j.strusafe.2006.07.010.

[Blatman2010]

Blatman G., Sudret B. (2010). An adaptive algorithm to build up sparse polynomial chaos expansions for stochastic finite element analysis. Probabilistic Engineering Mechanics, 25(2), Pages 183–197. doi: https://doi.org/10.1016/j.probengmech.2009.10.003.

[Eijnden2019]

van den Eijnden A.P., Schweckendiek T., Hicks M.A. (2019). Metamodelling for geotechnical reliability analysis with noisy and incomplete models. Submitted to Structural Safety.

Contributors & Sponsors¶

Contributors¶

Arpad Rozsas [*] [1] [2] [3] [4] [6] [7] [8] [9]
Arthur Slobbe [†] [1] [4] [5] [6] [8]
Shehryar Ali [‡] [2] [3]
Tianxiang Wang [§] [6] [7]
Jean-Marc Bourinet [¶] [5]
Liesette la Gasse [#] [4] [6] [7]
Rein de Vries [♠] [4] [6]
Juan Gonzalo Sepulveda Astudillo [♥] [5] [6]
Michael Havbro Faber [♦] [5]
Giulia Martini [♣] [4] [6] [7] [9]
Bram van den Eijnden [**] [5]

We are thankful to

Eric Bonet for helping with the initial setup of the web server hosting.
Xuzheng Chai, Giulia Martini, Nadieh Meinen, and Wim Courage for helping with testing the web server.
Agnieszka Bigaj van Vliet and Henk Miedema for supporting the project.

[1]	(1, 2) concept

[2]	(1, 2) web server design

[3]	(1, 2) web server implementation

[4]	(1, 2, 3, 4, 5) Sphinx documentation

[5]	(1, 2, 3, 4, 5) collecting reliability problems

[6]	(1, 2, 3, 4, 5, 6, 7) Python and/or Matlab implementation of the performance functions, LaTeX equations

[7]	(1, 2, 3, 4) visualization and plots

[8]	(1, 2) project management (securing budget, planning, task definition, etc.)

[9]	(1, 2) post-processing results

Note

If multiple persons contributed to a category, the references to each contributor appear in round brackets after the footnote number in square brackets.

[*]	arpad.rozsas@tno.nl

[†]	arthur.slobbe@tno.nl

[‡]	shehry3894@gmail.com

[§]	sysuwangtx@hotmail.com

[¶]	jean-marc.bourinet@sigma-clermont.fr

[#]	liesette.lagasse@tno.nl

[♠]	rein.devries@tno.nl

[♥]	jgsa@civil.aau.dk

[♦]	mfn@civil.aau.dk

[♣]	giulia.martini@tno.nl

[**]	A.P.vandenEijnden@tudelft.nl

Sponsors¶

The following organizations have generously contributed to the development and maintenance of the challenge framework and repository:

TNO: the development work was mainly financed by TNO, the working hours of those involved from TNO and Shehryar Ali’s were covered by TNO.
Diana: generously provided 10 professional Diana licences to for surrogating computatioanlly demanding numerical models.

Contact¶

If you have any questions or observations please get in touch via email or Git Issues. If you wish to contribute, you can do it directly to the GitLab repository following the standard git procedure.

Citation¶

If you use some code or results from this website please cite us:

Rozsas A., Slobbe A. (2019). Repository and Black-box Reliability Challenge 2019. https://gitlab.com/rozsasarpi/rprepo/.

Bug Reports & Questions¶

This project is MIT-licensed and the source is available on GitLab. If any questions or issues arise as you use it, please get in touch via Git Issues. If you wish to contribute, you can do it directly to the GitLab repository following the standard git procedure.

Sponsors ¶