Results¶

Thank you everyone who participated in, contributed to, and/or was interested in the Black-box Reliability Challenge 2019. As a first step of disseminating the results we:

made all reference solutions and performance functions public, see Reliability problems;
summarize the main results in the plots below (participants are anonymized) (Visual summary);
made the web server code (framework) publicly available.

If you use some code or results from this website please cite us (Citation).

If you have any suggestions and/or questions regarding the results or about the reliability challenge in general, do not hesitate to Contact us.

Visual summary¶

Note

To zoom into the images open them in a new tab or download them.

_images/beta_final_submission.facet.hidden_username.png

Fig. 1 Comparison of submitted system reliability indices to reference solutions (horizontal black lines).

_images/beta_vs_num_eval.facet.set1.hidden_username.png

Fig. 2 Set 1. Submitted system reliability indices and corresponding number of performance function evaluations in comparison to reference solutions (horizontal black lines).

_images/beta_vs_num_eval.facet.set2.hidden_username.png

Fig. 3 Set 2. Submitted system reliability indices and corresponding number of performance function evaluations in comparison to reference solutions (horizontal black lines).

Next steps¶

We organized a follow-up workshop that took place in Delft, the Netherlands on the 24-25th of February 2020. During the workshop the participants shared the details of their reliability algorithms and their experience for the benefit of the reliability community. For more information - including the presentations and summary of the workshop - visit the website of the workshop.

We plan to write up the main outcomes of the workshop into a brief report and make it available on the website of the workshop. Moreover, we plan to disseminate the results via journal paper(s).

Note

We are looking for volunteers to organize the next reliability challenge. If you are interested to lead or participate in the organization, have suggestions on how to improve and extend the challenge, and/or would like to donate reliability problems, please contact us.