Search-driven String Constraint Solving for Vulnerability Detection
1 like358 views
The document presents a search-driven approach to solving string constraints for vulnerability detection. State-of-the-art solvers like Z3-str2 have limitations in supporting complex string operations. The proposed approach decomposes constraints and leverages an automata-based solver to reduce the search space, before using a search-based solver to find satisfying assignments. An evaluation on 43 programs shows the approach significantly improves vulnerability detection effectiveness over baseline solvers, with affordable time costs. The automata-based solver plays a key role in the effectiveness of the search-based procedure.
1 of 35
Downloaded 10 times
![protected void authenticate() {
String user = req.getParameter("user"); // SOURCE
String pin = req.getParameter("pin"); // SOURCE
String token = req.getParameter("token"); // SOURCE
Document doc = db.parse(new File("users.xml"));
if(user.isEmpty() || pin.isEmpty() ||
!token.matches("[0-9]{8}")) {
// …
} else {
String q = "/users/user[@id='" +
ESAPI.encoder().encodeForXPath(user) +
"' and @pin=" +
ESAPI.encoder().encodeForXPath(pin) +
"]";
// …
NodeList nl=(NodeList)xpath.evaluate(q); // SINK
// …
}
}
3
A vulnerable example program](https://image.slidesharecdn.com/icse2017-170527031414/85/Search-driven-String-Constraint-Solving-for-Vulnerability-Detection-3-320.jpg)
![protected void authenticate() {
String user = req.getParameter("user");
String pin = req.getParameter("pin");
String token = req.getParameter("token");
Document doc = db.parse(new File("users.xml"));
if(user.isEmpty() || pin.isEmpty() ||
!token.matches("[0-9]{8}")) {
// …
} else {
String q = "/users/user[@id='" +
ESAPI.encoder().encodeForXPath(user) +
"' and @pin=" +
ESAPI.encoder().encodeForXPath(pin) +
"]";
// …
NodeList nl=(NodeList)xpath.evaluate(q);
// …
}
}
4
A vulnerable example program
"0 or 1"
"eve"
The program is vulnerable to
XPath Injection attacks
"/users/user[@id='evil' and
@pin=0 or 1]"](https://image.slidesharecdn.com/icse2017-170527031414/85/Search-driven-String-Constraint-Solving-for-Vulnerability-Detection-4-320.jpg)
![len(v0user
) > 0
len(v0pin
) > 0
v1user
= encodeForXPath(v0user
)
v0q
= concat("/users/user[@id='", v1user
)
v1q
= concat(v0q
, "' and @pin=")
v1pin
= encodeForXPath(v0pin
)
v2q
= concat(v1q
, v1pin
)
v3q
= concat(v2q
, "]")
matches(v1pin
, "[0-9]+ [Oo][Rr] 1=1")
matches(v0token
, "[0-9]{8}")
Attack Condition Decomposition
16
len(v0user
) > 0
len(v0pin
) > 0
matches(v0token
, "[0-9]{8}")
v1user
= encodeForXPath(v0user
)
v0q
= concat("/users/user[@id='", v1user
)
v1q
= concat(v0q
, "' and @pin=")
v1pin
= encodeForXPath(v0pin
)
v2q
= concat(v1q
, v1pin
)
v3q
= concat(v2q
, "]")
matches(v1pin
, "[0-9]+ [Oo][Rr] 1=1")](https://image.slidesharecdn.com/icse2017-170527031414/85/Search-driven-String-Constraint-Solving-for-Vulnerability-Detection-16-320.jpg)
![Attack Condition Decomposition
17
len(v0user
) > 0
len(v0pin
) > 0
v1user
= encodeForXPath(v0user
)
v0q
= concat("/users/user[@id='", v1user
)
v1q
= concat(v0q
, "' and @pin=")
v1pin
= encodeForXPath(v0pin
)
v2q
= concat(v1q
, v1pin
)
v3q
= concat(v2q
, "]")
matches(v1pin
, "[0-9]+ [Oo][Rr] 1=1")
len(v0user
) > 0
len(v0pin
) > 0
matches(v0token
, "[0-9]{8}")
v1user
= encodeForXPath(v0user
)
v0q
= concat("/users/user[@id='", v1user
)
v1q
= concat(v0q
, "' and @pin=")
v1pin
= encodeForXPath(v0pin
)
v2q
= concat(v1q
, v1pin
)
v3q
= concat(v2q
, "]")
matches(v1pin
, "[0-9]+ [Oo][Rr] 1=1")
matches(v0token
, "[0-9]{8}")](https://image.slidesharecdn.com/icse2017-170527031414/85/Search-driven-String-Constraint-Solving-for-Vulnerability-Detection-17-320.jpg)
![19
len(v0user
) > 0
len(v0pin
) > 0
v1user
= encodeForXPath(v0user
)
v0q
= concat("/users/user[@id='", v1user
)
v1q
= concat(v0q
, "' and @pin=")
v1pin
= encodeForXPath(v0pin
)
v2q
= concat(v1q
, v1pin
)
v3q
= concat(v2q
, "]")
matches(v1pin
, "[0-9]+ [Oo][Rr] 1=1")
matches(v0token
, "[0-9]{8}")
SAT/UNSAT/Crash
Attack Condition Partition
SAT/UNSAT/Crash
Invoke External Solver
Result](https://image.slidesharecdn.com/icse2017-170527031414/85/Search-driven-String-Constraint-Solving-for-Vulnerability-Detection-19-320.jpg)
![Invoke External Solver
20
matches(v0token
, "[0-9]{8}")
SAT
ResultAttack Condition Partition
Crash
len(v0user
) > 0
len(v0pin
) > 0
v1user
= encodeForXPath(v0user
)
v0q
= concat("/users/user[@id='", v1user
)
v1q
= concat(v0q
, "' and @pin=")
v1pin
= encodeForXPath(v0pin
)
v2q
= concat(v1q
, v1pin
)
v3q
= concat(v2q
, "]")
matches(v1pin
, "[0-9]+ [Oo][Rr] 1=1")](https://image.slidesharecdn.com/icse2017-170527031414/85/Search-driven-String-Constraint-Solving-for-Vulnerability-Detection-20-320.jpg)
Ad
Recommended
Applications of Machine Learning and Metaheuristic Search to Security Testing
Applications of Machine Learning and Metaheuristic Search to Security TestingLionel Briand This document discusses testing web application firewalls (WAFs) for SQL injection (SQLi) vulnerabilities. It states that the testing goal is to generate test cases that result in executable malicious SQL statements that can bypass the WAF. It also notes that WAF filter rules often need customization to avoid false positives and protect against new attacks, but that customization is error-prone due to complex rules, time/resource constraints, and a lack of automated tools.
System Testing of Timing Requirements based on Use Cases and Timed Automata
System Testing of Timing Requirements based on Use Cases and Timed AutomataLionel Briand The document describes a technique called TAUC that combines use case specifications and timed automata to automatically generate test cases for validating timing requirements of safety-critical systems. TAUC models the system functionality and environment as timed automata, identifies test inputs from use case scenarios to trigger state transitions, and employs a coverage-based and metaheuristic search approach to generate a test suite that stresses timing constraints. An evaluation on a case study shows TAUC achieves a 91% fault detection rate, significantly outperforming random and manual testing.
A Search-based Testing Approach for XML Injection Vulnerabilities in Web Appl...
A Search-based Testing Approach for XML Injection Vulnerabilities in Web Appl...Lionel Briand The document describes a search-based testing approach for detecting XML injection vulnerabilities in web applications. The approach uses genetic algorithms to search the input space to generate malicious XML outputs defined as test objectives (TOs). The approach was evaluated on four subjects and found to be highly effective at detecting vulnerabilities, achieving 100% TO coverage. Random search was not effective, covering zero TOs. The approach was efficient, taking 5-32 minutes per TO. Input validation decreased coverage while fewer inputs and a restricted alphabet increased efficiency.
Scalable Software Testing and Verification of Non-Functional Properties throu...
Scalable Software Testing and Verification of Non-Functional Properties throu...Lionel Briand This document discusses scalable software testing and verification of non-functional properties through heuristic search and optimization. It describes several projects with industry partners that use metaheuristic search techniques like hill climbing and genetic algorithms to generate test cases for non-functional properties of complex, configurable software systems. The techniques address issues of scalability and practicality for engineers by using dimensionality reduction, surrogate modeling, and dynamically adjusting the search strategy in different regions of the input space. The results provided worst-case scenarios more effectively than random testing alone.
Extracting Domain Models from Natural-Language Requirements: Approach and Ind...
Extracting Domain Models from Natural-Language Requirements: Approach and Ind...Lionel Briand 1) The document presents an approach for automatically extracting domain models from natural language requirements documents. It uses grammatical dependencies and novel extraction rules to construct domain models from requirement statements.
2) An empirical evaluation of the approach on three case studies found that generic extraction rules were triggered most frequently, while pattern-based rules were seldom triggered. Interviews on one case study found the extracted relations to be highly correct on average but only moderately relevant.
3) The conclusions indicate that the novel extraction rules hold practical significance but room remains to improve the relevance of automatically extracted domain models, such as by addressing common knowledge and natural language processing mistakes.
Testing of Cyber-Physical Systems: Diversity-driven Strategies
Testing of Cyber-Physical Systems: Diversity-driven StrategiesLionel Briand Lionel Briand discusses strategies for testing cyber-physical systems using diversity-driven approaches. He outlines challenges in verifying controllers and decision-making components in cyber-physical systems due to large input spaces and expensive model execution. Briand proposes maximizing diversity of test cases to improve fault detection. He describes using diversity of input signals, output signals, and failure patterns to generate test cases. Search algorithms are used to find test cases that maximize diversity or reveal specific failure patterns. The strategies are shown to significantly outperform coverage-based and random testing on Simulink models.
Incremental Reconfiguration of Product Specific Use Case Models for Evolving ...
Incremental Reconfiguration of Product Specific Use Case Models for Evolving ...Lionel Briand 1. The document discusses an approach for incrementally reconfiguring product-specific use case models when configuration decisions evolve over time.
2. Product-specific models are regenerated by focusing only on the changed decisions and their side effects, preserving unaffected parts and existing trace links.
3. The approach involves matching decision model elements before and after changes, calculating differences, and using these to reconfigure use case diagrams and specifications while generating an impact report.
Automated Vulnerability Testing Using Machine Learning and Metaheuristic Search
Automated Vulnerability Testing Using Machine Learning and Metaheuristic SearchLionel Briand The document discusses automated vulnerability testing of web applications using machine learning and metaheuristic search. It discusses three key areas:
1. Testing front-end web applications for XML injection vulnerabilities by automatically generating malicious XML messages and using search-based techniques to find input strings that allow the generation of these messages.
2. Testing web application firewalls (WAFs) using machine learning to learn attack patterns from successful and blocked attacks to generate new attacks, and a multi-objective genetic algorithm to automatically repair vulnerable WAF rule sets.
3. Using machine learning to detect malicious SQL statements at the database level by training on legitimate and attack SQL logs and classifying new statements as legitimate or attacks.
Search-Based Robustness Testing of Data Processing Systems
Search-Based Robustness Testing of Data Processing SystemsLionel Briand This document describes an approach to automatically generate robustness test cases for data processing systems using search-based testing and evolutionary algorithms. The approach uses model-based mutation to generate test inputs and model-based validation as an oracle to evaluate the test cases. An evolutionary algorithm is used to search for test cases that optimize coverage of various objectives like covering different parts of the data model, different types of data faults, clauses in input/output constraints, and code elements. The effectiveness of generated test suites is determined by measuring coverage of these objectives. The approach was evaluated on an industrial data processing system as a case study.
STAR: Stack Trace based Automatic Crash Reproduction
STAR: Stack Trace based Automatic Crash ReproductionSung Kim This document summarizes Ning Chen's PhD thesis defense on STAR, a stack trace based automatic crash reproduction framework. The outline includes motivation and related work, approaches of STAR including crash precondition computation and input model generation, evaluation study, challenges and future work, and contributions. STAR aims to efficiently reproduce crashes using only stack traces by performing backward symbolic execution to optimize crash precondition computation and address object creation challenges in reproducing object-oriented crashes.
Test Case Prioritization for Acceptance Testing of Cyber Physical Systems
Test Case Prioritization for Acceptance Testing of Cyber Physical SystemsLionel Briand 1) The document describes a multi-objective search-based approach for minimizing and prioritizing acceptance test cases for cyber physical systems to address challenges like time overhead, uncertainties in execution time, and risks of hardware damage.
2) The approach models acceptance tests, minimizes test cases by removing redundant operations, and prioritizes test cases using Monte Carlo simulation to estimate execution times while optimizing for criticality and risk.
3) An empirical evaluation on an industrial case study of in-orbit satellite testing shows the approach generates test suites that cover more test cases and have lower hardware risk within time budgets compared to manually created test suites.
Improving Fault Localization for Simulink Models using Search-Based Testing a...
Improving Fault Localization for Simulink Models using Search-Based Testing a...Lionel Briand This document summarizes an approach to improve fault localization for Simulink models using search-based testing and prediction models. The approach generates a small set of additional test cases to diversify an existing test suite using different test objectives. Predictor models are used to determine when adding more test cases is unlikely to improve fault localization rankings, allowing test generation to stop. An evaluation on 60 faulty industrial Simulink models found the approach significantly improved fault localization accuracy while reducing the number of new test cases generated by over half compared to generating test cases without the predictor models.
Automated Test Suite Generation for Time-Continuous Simulink Models
Automated Test Suite Generation for Time-Continuous Simulink ModelsLionel Briand This document summarizes an approach for automated test suite generation for Simulink models with time-continuous behaviors. It discusses two main challenges with existing Simulink testing techniques: incompatibility with the underlying SAT/SMT-based techniques which cannot handle features like time-continuous blocks, and low fault revealing ability when test oracles are manual. The proposed approach uses search-based test generation driven by output diversity and failure patterns to generate test cases that are more likely to reveal faults. An evaluation compares the fault detection capability of the approach to Simulink Design Verifier and finds that the proposed output diversity technique outperforms it. The approach is implemented in a tool called SimCoTest.
Effective Test Suites for ! Mixed Discrete-Continuous Stateflow Controllers
Effective Test Suites for ! Mixed Discrete-Continuous Stateflow ControllersLionel Briand The document describes algorithms for generating effective test suites for mixed discrete-continuous controllers modeled in Stateflow. It introduces the challenges of testing cyber-physical systems with both discrete and continuous behaviors. It then presents six test generation algorithms, including ones based on input diversity, state/transition coverage, and output diversity/stability/continuity. An evaluation of these algorithms on three industrial case studies examines their fault detection abilities, how they compare to each other, and how test suite size impacts results. The best performing algorithms focused on maximizing differences between output signals.
Log-Based Slicing for System-Level Test Cases
Log-Based Slicing for System-Level Test CasesLionel Briand DS3 is a technique that decomposes complex system test cases into simpler test cases using log-based program slicing. It performs the following steps:
1. Uses static slicing to initially decompose test cases based on assertions.
2. Analyzes test case execution logs to identify dependencies between statements and global resources.
3. Refines the static slices by adding missing dependencies identified in the logs.
4. Merges slices that share all non-assertion statements to minimize duplication.
An evaluation on two large systems found that DS3 produced well-formed test case slices and significantly reduced test case execution times compared to the original complex test cases and standard static slicing. The sliced test cases also maintained similar
Testing the Untestable: Model Testing of Complex Software-Intensive Systems
Testing the Untestable: Model Testing of Complex Software-Intensive SystemsLionel Briand This document discusses model testing as an approach to testing complex, software-intensive systems that are difficult or impossible to fully automate. It presents model testing as shifting the focus of testing from implemented systems to executable models that capture relevant system behavior and properties. Model testing aims to find and execute high-risk test scenarios in large input spaces and help guide targeted testing of implemented systems. Challenges include defining testable models that include dynamic and uncertain behavior, performing effective test selection, and detecting failures under uncertainty.
Testing Dynamic Behavior in Executable Software Models - Making Cyber-physica...
Testing Dynamic Behavior in Executable Software Models - Making Cyber-physica...Lionel Briand This document discusses testing dynamic behavior in executable software models for cyber-physical systems. It presents challenges for model-in-the-loop (MiL) testing due to large input spaces, expensive simulations, and lack of simple oracles. The document proposes using search-based testing to generate critical test cases by formulating it as a multi-objective optimization problem. It demonstrates the approach on an advanced driver assistance system and discusses improving performance with surrogate modeling.
Applying Product Line Use Case Modeling ! in an Industrial Automotive Embedde...
Applying Product Line Use Case Modeling ! in an Industrial Automotive Embedde...Lionel Briand 1. The document describes a refined approach to product line use case modeling called PUM that was applied in an automotive embedded system project at IEE.
2. PUM models variability in use case diagrams, specifications, and domain models using extensions to existing modeling artifacts like use case diagrams and restricted use case modeling.
3. The approach aims to support change management and impact analysis in product lines while limiting additional modeling overhead.
Heterogeneous Defect Prediction (
ESEC/FSE 2015)
Heterogeneous Defect Prediction (
ESEC/FSE 2015)Sung Kim This document describes a technique called Heterogeneous Defect Prediction (HDP) that aims to perform cross-project defect prediction even when the source and target projects have different sets of metrics (i.e. heterogeneous metrics). HDP first selects informative metrics from the source and target projects, then matches metrics between the projects that have similar distributions. It uses the matched metrics to build a prediction model on the source project and apply it to the target project. The technique is evaluated on multiple public defect datasets and is shown to outperform whole-project defect prediction and other cross-project defect prediction baselines in most cases, demonstrating the potential of HDP to reuse existing defect data more widely.
OCLR: A More Expressive, Pattern-Based Temporal Extension of OCL
OCLR: A More Expressive, Pattern-Based Temporal Extension of OCLLionel Briand This document introduces OCLR, a temporal extension of the Object Constraint Language (OCL) for expressing temporal properties. It discusses limitations of existing temporal logics and extensions of OCL for expressing temporal properties. It then presents the grammar and key features of OCLR, including support for Dwyer's pattern system of temporal property patterns (e.g. universality, existence, absence, response, precedence patterns) and precise specification of event scopes and distances between events. OCLR aims to provide a more expressive and pattern-based way to specify temporal properties within the model-driven engineering approach compared to existing temporal extensions of OCL.
Automated and Scalable Solutions for Software Testing: The Essential Role of ...
Automated and Scalable Solutions for Software Testing: The Essential Role of ...Lionel Briand 1) Modeling plays an essential role in enabling automated and scalable software testing solutions across many industrial domains like automotive, aerospace, and healthcare.
2) Models of requirements, system architecture, and environment behavior can be used to guide test generation, derive oracles, and enable early system testing through simulation.
3) Effective test automation solutions combine models with techniques like optimization, constraint solving, and natural language processing to address challenges of scalability, oracle generation, and exploring large test input spaces.
CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)
CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)Sung Kim CrashLocator is a technique that locates crashing faults based solely on crash stack traces, without needing test cases or instrumentation. It approximates failing traces from crash stacks using static analysis techniques like control flow analysis and backward slicing. It then ranks suspicious functions based on characteristics of faulty functions, such as frequency in crash traces and distance from crash point. An evaluation on Mozilla projects found CrashLocator could locate over 50% of faults in the top result and over 63% in the top 5 results, outperforming conventional stack-only methods.
Combining genetic algoriths and constraint programming to support stress test...
Combining genetic algoriths and constraint programming to support stress test...Lionel Briand The document presents a search strategy called GA+CP that combines genetic algorithms and constraint programming to identify scenarios likely to violate task deadlines in real-time embedded systems. GA+CP casts the generation of stress test cases as an optimization problem to find arrival times that maximize the chance of deadline violations. GA is efficient and generates diverse test cases, while CP is effective at finding test cases more likely to violate deadlines. The approach uses GA to evolve potential solutions and CP to improve the most promising ones in each generation.
AN EMPIRICAL STUDY ON THE POTENTIAL USEFULNESS OF DOMAIN MODELS FOR COMPLETEN...
AN EMPIRICAL STUDY ON THE POTENTIAL USEFULNESS OF DOMAIN MODELS FOR COMPLETEN...Lionel Briand The study empirically examined the potential usefulness of domain models for detecting incompleteness in natural language requirements (RQ1). Domain models were found to provide useful hints towards requirements omissions, with higher sensitivity to unspecified versus underspecified requirements. Additionally, the sensitivity of domain models in detecting omissions varied based on the intrinsic properties of requirements documents, such as the frequency of key phrases (RQ2). Specifically, key phrases in requirements provided an accurate way to predict domain model sensitivity to omissions without full analysis. Therefore, domain models show promise as an external source for requirements completeness checking.
Mining Assumptions for Software Components using Machine Learning
Mining Assumptions for Software Components using Machine LearningLionel Briand EPIcuRus is an approach to automatically generate assumptions for software components in cyber-physical systems modeled in Simulink. It uses machine learning techniques to mine assumptions from test case results. The approach includes generating tests cases using an important feature boundary test generation method, model checking candidate assumptions, and selecting the most informative safe assumptions. An evaluation on industrial case studies found it can learn non-vacuous assumptions for most requirements within a practical time limit, with the important feature boundary test generation performing best.
SSBSE 2020 keynote
SSBSE 2020 keynoteShiva Nejati This document discusses search-based testing and its applications in software testing. It outlines some key strengths of search-based software testing (SBST) such as being scalable, parallelizable, versatile, and flexible. It also discusses some limitations of search-based approaches for problems that require formal verification to establish properties for all possible usages. The document compares classical optimization approaches, which build solutions incrementally, to stochastic optimization approaches used in SBST, which sample solutions in a randomized way. It notes that while testing can find bugs, it cannot prove their absence. Finally, it discusses how SBST can be combined with other techniques like constraint solving and machine learning.
Known XML Vulnerabilities Are Still a Threat to Popular Parsers ! & Open Sour...
Known XML Vulnerabilities Are Still a Threat to Popular Parsers ! & Open Sour...Lionel Briand The document summarizes research into vulnerabilities in XML parsers related to billion laughs (BIL) denial of service attacks and XML external entity (XXE) attacks. The research assessed 13 XML parsers and 8 open source systems that use XML parsers. It found that over half of the parsers tested were vulnerable to BIL and XXE attacks. It also found that all of the open source systems tested were vulnerable because they used vulnerable parsers without applying any mitigation techniques. The research concludes that BIL and XXE attacks remain successful against many modern XML parsers and systems that use them. It recommends that software developers configure parsers securely and limit vulnerable features, and that parser developers improve security in their default configurations.
A Natural Language Programming Approach for Requirements-based Security Testing
A Natural Language Programming Approach for Requirements-based Security TestingLionel Briand The document describes a natural language programming approach called Misuse Case Programming (MCP) to automatically generate executable security test cases from requirements specified using misuse cases. MCP maps misuse case specifications written in natural language to test code by identifying inputs, operations, and conditions and generating code with control structures, variable declarations, and method calls. The approach was applied to requirements for a web-based, multi-device healthcare system to automatically test for vulnerabilities like authorization bypass.
Georgy Nosenko - An introduction to the use SMT solvers for software security
Georgy Nosenko - An introduction to the use SMT solvers for software securityDefconRussia The document discusses how SMT solvers can be used for software security applications such as bug hunting, exploit generation, protection analysis, and malware analysis by modeling portions of code or algorithms as logical formulas that can then be analyzed using an SMT solver to prove properties or generate inputs. It provides examples of how SMT solvers have been used to find integer overflows, help with program verification, and aid in defeating simple hashing algorithms.
Building High-Performance Language Implementations With Low Effort
Building High-Performance Language Implementations With Low EffortStefan Marr The document discusses building high-performance language implementations with minimal effort, highlighting the importance of understanding programming languages for improved performance and productivity. It covers various language implementation strategies such as interpreters and compilers, and introduces techniques like Just-In-Time (JIT) compilation and meta-tracing to enhance execution speed. Additionally, it emphasizes self-optimizing techniques through node specialization and other optimizations for efficient programming language execution.
More Related Content
What's hot (20)
Search-Based Robustness Testing of Data Processing Systems
Search-Based Robustness Testing of Data Processing SystemsLionel Briand This document describes an approach to automatically generate robustness test cases for data processing systems using search-based testing and evolutionary algorithms. The approach uses model-based mutation to generate test inputs and model-based validation as an oracle to evaluate the test cases. An evolutionary algorithm is used to search for test cases that optimize coverage of various objectives like covering different parts of the data model, different types of data faults, clauses in input/output constraints, and code elements. The effectiveness of generated test suites is determined by measuring coverage of these objectives. The approach was evaluated on an industrial data processing system as a case study.
STAR: Stack Trace based Automatic Crash Reproduction
STAR: Stack Trace based Automatic Crash ReproductionSung Kim This document summarizes Ning Chen's PhD thesis defense on STAR, a stack trace based automatic crash reproduction framework. The outline includes motivation and related work, approaches of STAR including crash precondition computation and input model generation, evaluation study, challenges and future work, and contributions. STAR aims to efficiently reproduce crashes using only stack traces by performing backward symbolic execution to optimize crash precondition computation and address object creation challenges in reproducing object-oriented crashes.
Test Case Prioritization for Acceptance Testing of Cyber Physical Systems
Test Case Prioritization for Acceptance Testing of Cyber Physical SystemsLionel Briand 1) The document describes a multi-objective search-based approach for minimizing and prioritizing acceptance test cases for cyber physical systems to address challenges like time overhead, uncertainties in execution time, and risks of hardware damage.
2) The approach models acceptance tests, minimizes test cases by removing redundant operations, and prioritizes test cases using Monte Carlo simulation to estimate execution times while optimizing for criticality and risk.
3) An empirical evaluation on an industrial case study of in-orbit satellite testing shows the approach generates test suites that cover more test cases and have lower hardware risk within time budgets compared to manually created test suites.
Improving Fault Localization for Simulink Models using Search-Based Testing a...
Improving Fault Localization for Simulink Models using Search-Based Testing a...Lionel Briand This document summarizes an approach to improve fault localization for Simulink models using search-based testing and prediction models. The approach generates a small set of additional test cases to diversify an existing test suite using different test objectives. Predictor models are used to determine when adding more test cases is unlikely to improve fault localization rankings, allowing test generation to stop. An evaluation on 60 faulty industrial Simulink models found the approach significantly improved fault localization accuracy while reducing the number of new test cases generated by over half compared to generating test cases without the predictor models.
Automated Test Suite Generation for Time-Continuous Simulink Models
Automated Test Suite Generation for Time-Continuous Simulink ModelsLionel Briand This document summarizes an approach for automated test suite generation for Simulink models with time-continuous behaviors. It discusses two main challenges with existing Simulink testing techniques: incompatibility with the underlying SAT/SMT-based techniques which cannot handle features like time-continuous blocks, and low fault revealing ability when test oracles are manual. The proposed approach uses search-based test generation driven by output diversity and failure patterns to generate test cases that are more likely to reveal faults. An evaluation compares the fault detection capability of the approach to Simulink Design Verifier and finds that the proposed output diversity technique outperforms it. The approach is implemented in a tool called SimCoTest.
Effective Test Suites for ! Mixed Discrete-Continuous Stateflow Controllers
Effective Test Suites for ! Mixed Discrete-Continuous Stateflow ControllersLionel Briand The document describes algorithms for generating effective test suites for mixed discrete-continuous controllers modeled in Stateflow. It introduces the challenges of testing cyber-physical systems with both discrete and continuous behaviors. It then presents six test generation algorithms, including ones based on input diversity, state/transition coverage, and output diversity/stability/continuity. An evaluation of these algorithms on three industrial case studies examines their fault detection abilities, how they compare to each other, and how test suite size impacts results. The best performing algorithms focused on maximizing differences between output signals.
Log-Based Slicing for System-Level Test Cases
Log-Based Slicing for System-Level Test CasesLionel Briand DS3 is a technique that decomposes complex system test cases into simpler test cases using log-based program slicing. It performs the following steps:
1. Uses static slicing to initially decompose test cases based on assertions.
2. Analyzes test case execution logs to identify dependencies between statements and global resources.
3. Refines the static slices by adding missing dependencies identified in the logs.
4. Merges slices that share all non-assertion statements to minimize duplication.
An evaluation on two large systems found that DS3 produced well-formed test case slices and significantly reduced test case execution times compared to the original complex test cases and standard static slicing. The sliced test cases also maintained similar
Testing the Untestable: Model Testing of Complex Software-Intensive Systems
Testing the Untestable: Model Testing of Complex Software-Intensive SystemsLionel Briand This document discusses model testing as an approach to testing complex, software-intensive systems that are difficult or impossible to fully automate. It presents model testing as shifting the focus of testing from implemented systems to executable models that capture relevant system behavior and properties. Model testing aims to find and execute high-risk test scenarios in large input spaces and help guide targeted testing of implemented systems. Challenges include defining testable models that include dynamic and uncertain behavior, performing effective test selection, and detecting failures under uncertainty.
Testing Dynamic Behavior in Executable Software Models - Making Cyber-physica...
Testing Dynamic Behavior in Executable Software Models - Making Cyber-physica...Lionel Briand This document discusses testing dynamic behavior in executable software models for cyber-physical systems. It presents challenges for model-in-the-loop (MiL) testing due to large input spaces, expensive simulations, and lack of simple oracles. The document proposes using search-based testing to generate critical test cases by formulating it as a multi-objective optimization problem. It demonstrates the approach on an advanced driver assistance system and discusses improving performance with surrogate modeling.
Applying Product Line Use Case Modeling ! in an Industrial Automotive Embedde...
Applying Product Line Use Case Modeling ! in an Industrial Automotive Embedde...Lionel Briand 1. The document describes a refined approach to product line use case modeling called PUM that was applied in an automotive embedded system project at IEE.
2. PUM models variability in use case diagrams, specifications, and domain models using extensions to existing modeling artifacts like use case diagrams and restricted use case modeling.
3. The approach aims to support change management and impact analysis in product lines while limiting additional modeling overhead.
Heterogeneous Defect Prediction (
ESEC/FSE 2015)
Heterogeneous Defect Prediction (
ESEC/FSE 2015)Sung Kim This document describes a technique called Heterogeneous Defect Prediction (HDP) that aims to perform cross-project defect prediction even when the source and target projects have different sets of metrics (i.e. heterogeneous metrics). HDP first selects informative metrics from the source and target projects, then matches metrics between the projects that have similar distributions. It uses the matched metrics to build a prediction model on the source project and apply it to the target project. The technique is evaluated on multiple public defect datasets and is shown to outperform whole-project defect prediction and other cross-project defect prediction baselines in most cases, demonstrating the potential of HDP to reuse existing defect data more widely.
OCLR: A More Expressive, Pattern-Based Temporal Extension of OCL
OCLR: A More Expressive, Pattern-Based Temporal Extension of OCLLionel Briand This document introduces OCLR, a temporal extension of the Object Constraint Language (OCL) for expressing temporal properties. It discusses limitations of existing temporal logics and extensions of OCL for expressing temporal properties. It then presents the grammar and key features of OCLR, including support for Dwyer's pattern system of temporal property patterns (e.g. universality, existence, absence, response, precedence patterns) and precise specification of event scopes and distances between events. OCLR aims to provide a more expressive and pattern-based way to specify temporal properties within the model-driven engineering approach compared to existing temporal extensions of OCL.
Automated and Scalable Solutions for Software Testing: The Essential Role of ...
Automated and Scalable Solutions for Software Testing: The Essential Role of ...Lionel Briand 1) Modeling plays an essential role in enabling automated and scalable software testing solutions across many industrial domains like automotive, aerospace, and healthcare.
2) Models of requirements, system architecture, and environment behavior can be used to guide test generation, derive oracles, and enable early system testing through simulation.
3) Effective test automation solutions combine models with techniques like optimization, constraint solving, and natural language processing to address challenges of scalability, oracle generation, and exploring large test input spaces.
CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)
CrashLocator: Locating Crashing Faults Based on Crash Stacks (ISSTA 2014)Sung Kim CrashLocator is a technique that locates crashing faults based solely on crash stack traces, without needing test cases or instrumentation. It approximates failing traces from crash stacks using static analysis techniques like control flow analysis and backward slicing. It then ranks suspicious functions based on characteristics of faulty functions, such as frequency in crash traces and distance from crash point. An evaluation on Mozilla projects found CrashLocator could locate over 50% of faults in the top result and over 63% in the top 5 results, outperforming conventional stack-only methods.
Combining genetic algoriths and constraint programming to support stress test...
Combining genetic algoriths and constraint programming to support stress test...Lionel Briand The document presents a search strategy called GA+CP that combines genetic algorithms and constraint programming to identify scenarios likely to violate task deadlines in real-time embedded systems. GA+CP casts the generation of stress test cases as an optimization problem to find arrival times that maximize the chance of deadline violations. GA is efficient and generates diverse test cases, while CP is effective at finding test cases more likely to violate deadlines. The approach uses GA to evolve potential solutions and CP to improve the most promising ones in each generation.
AN EMPIRICAL STUDY ON THE POTENTIAL USEFULNESS OF DOMAIN MODELS FOR COMPLETEN...
AN EMPIRICAL STUDY ON THE POTENTIAL USEFULNESS OF DOMAIN MODELS FOR COMPLETEN...Lionel Briand The study empirically examined the potential usefulness of domain models for detecting incompleteness in natural language requirements (RQ1). Domain models were found to provide useful hints towards requirements omissions, with higher sensitivity to unspecified versus underspecified requirements. Additionally, the sensitivity of domain models in detecting omissions varied based on the intrinsic properties of requirements documents, such as the frequency of key phrases (RQ2). Specifically, key phrases in requirements provided an accurate way to predict domain model sensitivity to omissions without full analysis. Therefore, domain models show promise as an external source for requirements completeness checking.
Mining Assumptions for Software Components using Machine Learning
Mining Assumptions for Software Components using Machine LearningLionel Briand EPIcuRus is an approach to automatically generate assumptions for software components in cyber-physical systems modeled in Simulink. It uses machine learning techniques to mine assumptions from test case results. The approach includes generating tests cases using an important feature boundary test generation method, model checking candidate assumptions, and selecting the most informative safe assumptions. An evaluation on industrial case studies found it can learn non-vacuous assumptions for most requirements within a practical time limit, with the important feature boundary test generation performing best.
SSBSE 2020 keynote
SSBSE 2020 keynoteShiva Nejati This document discusses search-based testing and its applications in software testing. It outlines some key strengths of search-based software testing (SBST) such as being scalable, parallelizable, versatile, and flexible. It also discusses some limitations of search-based approaches for problems that require formal verification to establish properties for all possible usages. The document compares classical optimization approaches, which build solutions incrementally, to stochastic optimization approaches used in SBST, which sample solutions in a randomized way. It notes that while testing can find bugs, it cannot prove their absence. Finally, it discusses how SBST can be combined with other techniques like constraint solving and machine learning.
Known XML Vulnerabilities Are Still a Threat to Popular Parsers ! & Open Sour...
Known XML Vulnerabilities Are Still a Threat to Popular Parsers ! & Open Sour...Lionel Briand The document summarizes research into vulnerabilities in XML parsers related to billion laughs (BIL) denial of service attacks and XML external entity (XXE) attacks. The research assessed 13 XML parsers and 8 open source systems that use XML parsers. It found that over half of the parsers tested were vulnerable to BIL and XXE attacks. It also found that all of the open source systems tested were vulnerable because they used vulnerable parsers without applying any mitigation techniques. The research concludes that BIL and XXE attacks remain successful against many modern XML parsers and systems that use them. It recommends that software developers configure parsers securely and limit vulnerable features, and that parser developers improve security in their default configurations.
A Natural Language Programming Approach for Requirements-based Security Testing
A Natural Language Programming Approach for Requirements-based Security TestingLionel Briand The document describes a natural language programming approach called Misuse Case Programming (MCP) to automatically generate executable security test cases from requirements specified using misuse cases. MCP maps misuse case specifications written in natural language to test code by identifying inputs, operations, and conditions and generating code with control structures, variable declarations, and method calls. The approach was applied to requirements for a web-based, multi-device healthcare system to automatically test for vulnerabilities like authorization bypass.
Similar to Search-driven String Constraint Solving for Vulnerability Detection (20)
Georgy Nosenko - An introduction to the use SMT solvers for software security
Georgy Nosenko - An introduction to the use SMT solvers for software securityDefconRussia The document discusses how SMT solvers can be used for software security applications such as bug hunting, exploit generation, protection analysis, and malware analysis by modeling portions of code or algorithms as logical formulas that can then be analyzed using an SMT solver to prove properties or generate inputs. It provides examples of how SMT solvers have been used to find integer overflows, help with program verification, and aid in defeating simple hashing algorithms.
Building High-Performance Language Implementations With Low Effort
Building High-Performance Language Implementations With Low EffortStefan Marr The document discusses building high-performance language implementations with minimal effort, highlighting the importance of understanding programming languages for improved performance and productivity. It covers various language implementation strategies such as interpreters and compilers, and introduces techniques like Just-In-Time (JIT) compilation and meta-tracing to enhance execution speed. Additionally, it emphasizes self-optimizing techniques through node specialization and other optimizations for efficient programming language execution.
StatsCraft 2015: Monitoring using riemann - Moshe Zada
StatsCraft 2015: Monitoring using riemann - Moshe ZadaStatsCraft This document discusses using Riemann for event processing and alerting. It provides examples of using Riemann to implement basic alerts, visualize events by streaming to ELK, and aggregate events. It also covers techniques like adding throttling to alerts, ignoring spikes, and putting the system in maintenance mode to ignore alerts temporarily. Code examples are provided for building alert streams and processing events in Riemann.
Analyzing the Performance Effects of Meltdown + Spectre on Apache Spark Workl...
Analyzing the Performance Effects of Meltdown + Spectre on Apache Spark Workl...Databricks The document analyzes performance degradation of 3-5% in systems affected by Meltdown and Spectre vulnerabilities, detailing the mechanisms behind these exploits and their mitigations. It uses TPC-DS benchmarks to evaluate system performance and introduces protective strategies such as page table isolation and retpoline for indirect branch protection. The findings indicate a significant impact on CPU utilization and performance metrics across different setups.
PVS-Studio, a solution for resource intensive applications development
PVS-Studio, a solution for resource intensive applications developmentOOO "Program Verification Systems" PVS-Studio is a static code analyzer designed to identify errors in C, C++, and C# source code, incorporating rule sets covering general diagnostics, optimizations, and 64-bit error detection. The software supports integration with Visual Studio and various continuous integration systems, offering features like incremental analysis and filtering of messages. It detects a broad range of errors including copy-paste mistakes, buffer overflows, and issues related to 32-bit to 64-bit application migration.
How to write clean & testable code without losing your mind
How to write clean & testable code without losing your mindAndreas Czakaj The document outlines a presentation on how to write clean and testable code, particularly within the context of Adobe Experience Manager (AEM). It discusses the benefits of Test Driven Development (TDD), the importance of adhering to clean code principles such as the Single Responsibility Principle, and strategies for managing dependencies effectively. The speaker shares insights from their experiences with AEM development, emphasizing that a structured approach to TDD can enhance code reliability, test coverage, and overall developer efficiency.
Csw2016 gawlik bypassing_differentdefenseschemes
Csw2016 gawlik bypassing_differentdefenseschemesCanSecWest The document discusses crash-resistance in software and how it can be exploited. It explains how exceptions generated by crashes in callback functions in Windows are handled, allowing programs to continue running despite crashes. This crash-resistance property is demonstrated through a simple example program. The document then discusses how crash-resistant probing of memory can be used to bypass defenses like ASLR by scanning process memory from a web worker without crashing the browser. Techniques like heap spraying and type confusion are used to craft fake objects and scan memory in a crash-resistant manner to discover information like the TEB and DLL base addresses.
IBM IMPACT 2014 - AMC-1883 - Where's My Message - Analyze IBM WebSphere MQ Re...
IBM IMPACT 2014 - AMC-1883 - Where's My Message - Analyze IBM WebSphere MQ Re...Peter Broadhurst The document details various techniques for analyzing IBM WebSphere MQ recovery logs and troubleshooting messaging issues, including connectivity, message flow, and application behavior. It describes commands and monitoring tools, such as amqsclm for cluster queue monitoring, and emphasizes the importance of real-time monitoring of queues and channels. Additionally, it addresses common symptoms of messaging problems and provides guidelines for checking channel and queue statuses to improve system performance.
Designing Modern Streaming Data Applications
Designing Modern Streaming Data ApplicationsArun Kejariwal The document discusses the design and architecture of modern streaming data applications, focusing on the importance of real-time data processing for data-driven decision making. It highlights Apache Pulsar as a flexible, high-performance messaging system with features such as durable log storage, seamless cluster expansion, and built-in state management for serverless functions. Additionally, it contrasts Pulsar with other messaging systems like Kafka, emphasizing its scalability and support for multi-tenancy.
Spectre(v1%2 fv2%2fv4) v.s. meltdown(v3)
Spectre(v1%2 fv2%2fv4) v.s. meltdown(v3)Gavin Guo The document provides a detailed technical analysis of the Spectre and Meltdown vulnerabilities in computer systems, particularly focusing on their variants, impact, and mitigation strategies. It explains how these vulnerabilities exploit speculative execution in processors, allowing unauthorized access to sensitive data. The text also discusses various mitigation techniques, including code patches and the introduction of observable speculation barriers to prevent speculative execution.
CodeChecker summary 21062021
CodeChecker summary 21062021Olivera Milenkovic CodeChecker is a static analysis tool that finds bugs in code by performing interprocedural analysis across files and functions. It uses the Clang compiler infrastructure to run checks from tools like the Clang Static Analyzer and Clang Tidy. CodeChecker provides features like cross-translation unit analysis, suppressing and tagging issues, and integration with version control systems. It outputs analysis results that are easy for developers to understand and fix issues.
Ml5 svm and-kernels
Ml5 svm and-kernelsankit_ppt The document discusses support vector machines (SVMs). It explains that SVMs find the optimal separating hyperplane that maximizes the margin between two classes of data points. SVMs can handle both linear and non-linear classification problems by using kernels to implicitly map inputs into high-dimensional feature spaces. Common kernels include the radial basis function kernel, which transforms the data into an infinite dimensional space non-linearly.
Drd secr final1_3
Drd secr final1_3Devexperts This document summarizes a presentation on dynamic data race detection in concurrent Java programs. It discusses what data races are and why they are dangerous. It then covers different approaches to automatic race detection, including static and dynamic methods. The presentation focuses on explaining the happens-before race detection algorithm using vector clocks. It also describes the implementation of the presenters' own dynamic race detector, including how it works, problems they solved, and examples of races it detected in real applications.
Certification
Certificationsubhransu mishra The document describes the implementation of the Ad-hoc On-Demand Distance Vector (AODV) routing protocol in the Network Simulator 2 (NS-2). It discusses the file dependencies of AODV, the general flow of AODV operation through an example, the trace format used in NS-2, and some of the main implementation files and functions in AODV including timers, routing table management, and packet handling functions.
Cassandra 2.1 boot camp, Overview
Cassandra 2.1 boot camp, OverviewJoshua McKenzie The document describes the architecture of Cassandra, including its startup process, layers (API, Dynamo, database), services (StorageProxy, MessagingService, Gossip), and failure handling. It also details how nodes communicate via the Gossip protocol to exchange state information and maintain a consistent cluster view.
Battista Biggio @ AISec 2013 - Is Data Clustering in Adversarial Settings Sec...
Battista Biggio @ AISec 2013 - Is Data Clustering in Adversarial Settings Sec...Pluribus One The document presents a framework for evaluating the security of data clustering algorithms against adversarial attacks that may mislead clustering processes, especially in security-sensitive applications like malware detection. It defines potential attacks, such as poisoning and obfuscation, and evaluates their impacts on clustering integrity and availability. The work highlights vulnerabilities of clustering algorithms, particularly single-linkage hierarchical clustering, and suggests future research directions for developing more secure clustering methods.
alexnet.pdf
alexnet.pdfBhautikDaxini1 The document describes the AlexNet neural network architecture and its application to classifying images from the Fashion-MNIST dataset. It constructs an AlexNet model, loads and preprocesses the Fashion-MNIST data, and trains the model on this dataset for 5 epochs. Key aspects covered include the convolutional and pooling layers in AlexNet, reading and transforming the Fashion-MNIST data, calculating training and test accuracy, and observing slower progress during training compared to LeNet due to the larger image size.
MongoDB World 2019: Life In Stitch-es
MongoDB World 2019: Life In Stitch-esMongoDB The document outlines the team and services involved in Acxiom's measurement and analytics integration, detailing roles of key personnel and various strategies including testing, logging, and AWS integration. It emphasizes continuous improvement through testing and monitoring of APIs, as well as outlines the proxy capabilities and logging mechanisms. Albert Einstein quotes are interspersed throughout to highlight the importance of curiosity, questioning, and simplifying complexity in processes.
Android RenderScript on LLVM
Android RenderScript on LLVMJohn Lee The document discusses RenderScript on LLVM. It describes RenderScript as a way to perform 3D rendering and compute tasks portably and with high performance on Android. It outlines the main components: an offline compiler that optimizes scripts, an online JIT compiler, and a runtime library. It provides an example of using RenderScript to convert an image to grayscale and discusses how scripts are compiled and executed to provide fast launch times.
Static analysis: looking for errors ... and vulnerabilities?
Static analysis: looking for errors ... and vulnerabilities? Andrey Karpov The document discusses the significance of static analysis in identifying programming vulnerabilities in C and C++ code, highlighting that thousands of vulnerabilities are reported annually and most stem from simple programming errors. It emphasizes that early detection of issues can significantly reduce the cost of fixing them over time and outlines the advantages and disadvantages of static analysis as a quality assurance method. Additionally, it provides insights into the nature of vulnerabilities and stresses the importance of integrating static analysis into development workflows to enhance software security.
PVS-Studio, a solution for resource intensive applications development
PVS-Studio, a solution for resource intensive applications developmentOOO "Program Verification Systems"
Ad
More from Lionel Briand (20)
LTM: Scalable and Black-box Similarity-based Test Suite Minimization based on...
LTM: Scalable and Black-box Similarity-based Test Suite Minimization based on...Lionel Briand FSE 2025 presentation
TEASMA: A Practical Methodology for Test Adequacy Assessment of Deep Neural N...
TEASMA: A Practical Methodology for Test Adequacy Assessment of Deep Neural N...Lionel Briand FSE 2025 presentation
Automated Testing and Safety Analysis of Deep Neural Networks
Automated Testing and Safety Analysis of Deep Neural NetworksLionel Briand Keynote address at Internetware 2025, co-located with FSE 2025
FlakyFix: Using Large Language Models for Predicting Flaky Test Fix Categorie...
FlakyFix: Using Large Language Models for Predicting Flaky Test Fix Categorie...Lionel Briand Journal-first presentation at ICST 2025
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...Lionel Briand Keynote at RAISE workshop, ICSE 2025
Precise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalLionel Briand The document discusses the challenges of achieving precise and complete requirements upfront in software development projects. It notes that while academics assume detailed requirements are needed, practitioners find this difficult to achieve in reality due to limited resources, uncertainty, and changing needs. The document provides perspectives from practice that emphasize starting with prototypes and visions rather than detailed specifications. It also summarizes research finding diverse requirements practices across different domains and organizations. The document concludes that while precise requirements may be desirable, they are often elusive goals, and the focus should be on achieving compliance and delivering working software.
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLionel Briand Large language models show promise for test case repair tasks. LLMs can be applied to tasks like test case generation, classification of flaky tests, and test case evolution and repair. The paper presents TaRGet, a framework that uses LLMs for automated test case repair. TaRGet takes as input a broken test case and code changes to the system under test, and outputs a repaired test case. Evaluation shows TaRGet achieves over 80% plausible repair accuracy. The paper analyzes repair characteristics, evaluates different LLM and input/output formats, and examines the impact of fine-tuning data size on performance.
Metamorphic Testing for Web System Security
Metamorphic Testing for Web System SecurityLionel Briand This document summarizes a presentation on metamorphic testing for web system security given by Nazanin Bayati on September 13, 2023. Metamorphic testing uses relations between the outputs of multiple test executions to test systems when specifying expected outputs is difficult. It was applied to web systems by generating follow-up inputs based on transformations of valid interactions and checking that output relations held. The approach detected over 60% of vulnerabilities in tested systems and addressed more vulnerability types than static and dynamic analysis tools. It provides an effective and automated way to test for security issues in web systems.
Simulator-based Explanation and Debugging of Hazard-triggering Events in DNN-...
Simulator-based Explanation and Debugging of Hazard-triggering Events in DNN-...Lionel Briand This document proposes a method called SEDE (Simulator-based Explanations for DNN Errors) to automatically generate explanations for errors in DNN-based safety-critical systems by constraining simulator parameters. SEDE first identifies clusters of error-inducing images, then uses an evolutionary algorithm to generate simulator images within each cluster, including failing, passing, and representative images. SEDE extracts rules characterizing the unsafe parameter space and uses the generated images to retrain DNNs, improving accuracy compared to alternative methods. The paper evaluates SEDE on head pose and face landmark detection DNNs in terms of generating diverse cluster images, delimiting unsafe spaces, and enhancing DNN performance.
Fuzzing for CPS Mutation Testing
Fuzzing for CPS Mutation TestingLionel Briand This document summarizes a research paper on using grey-box fuzzing (MOTIF) for mutation testing of C/C++ code in cyber-physical systems (CPS). It introduces mutation testing and grey-box fuzzing, and proposes MOTIF which generates a fuzzing driver to test functions with live mutants. An empirical evaluation compares MOTIF to symbolic execution-based mutation testing on three subject programs. MOTIF killed more mutants within 10,000 seconds and was able to test programs that symbolic execution could not handle due to limitations like floating-point values. Seed inputs alone killed few mutants, showing the importance of fuzzing. MOTIF is an effective approach for mutation testing of CPS software.
Data-driven Mutation Analysis for Cyber-Physical Systems
Data-driven Mutation Analysis for Cyber-Physical SystemsLionel Briand Data-driven mutation analysis is proposed to assess if test suites for cyber-physical systems properly exercise component interoperability. Fault models are developed for different data types and dependencies, and are used to automatically generate mutants by injecting faults. Empirical results on industrial systems demonstrate the feasibility and effectiveness of the approach in identifying test suite shortcomings and poor oracles.
Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems
Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled SystemsLionel Briand This document proposes MORLOT (Many-Objective Reinforcement Learning for Online Testing) to address challenges in online testing of DNN-enabled systems. MORLOT leverages many-objective search and reinforcement learning to choose test actions. It was evaluated on the Transfuser autonomous driving system in the CARLA simulator using 6 safety requirements. MORLOT was significantly more effective and efficient at finding safety violations than random search or other many-objective approaches, achieving a higher average test effectiveness for any given test budget.
ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolu...
ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolu...Lionel Briand 1. The document presents ATM, a new approach for black-box test case minimization that transforms test code into abstract syntax trees and uses tree-based similarity measures and genetic algorithms to minimize test suites.
2. ATM was evaluated on the DEFECTS4J dataset and achieved a fault detection rate of 0.82 on average, significantly outperforming existing techniques, while requiring only practical execution times.
3. The best configuration of ATM used a genetic algorithm with a combined similarity measure, achieving a fault detection rate of 0.80 within 1.2 hours on average.
Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction ...
Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction ...Lionel Briand The document is a journal paper that proposes a method for black-box safety analysis and retraining of deep neural networks (DNNs) based on feature extraction and clustering of failure-inducing images. The method uses a pre-trained VGG16 model to extract features from failure images, clusters the features using DBSCAN, selects clusters that likely caused failures, and retrains the DNN to improve safety based on images in problematic clusters. An empirical evaluation on various DNNs for tasks like gaze detection showed the method effectively determined failure causes through clustering and improved models with fewer images than other approaches.
PRINS: Scalable Model Inference for Component-based System Logs
PRINS: Scalable Model Inference for Component-based System LogsLionel Briand PRINS is a technique for scalable model inference of component-based system logs. It divides the problem into inferring individual component models and then stitching them together. The paper evaluates PRINS on several systems and compares its execution time and accuracy to MINT, a state-of-the-art model inference tool. Results show that PRINS is significantly faster than MINT, especially on larger logs, with comparable accuracy. However, stitching component models can result in larger overall system models. The paper contributes an empirical evaluation of the PRINS technique and makes its implementation publicly available.
Revisiting the Notion of Diversity in Software Testing
Revisiting the Notion of Diversity in Software TestingLionel Briand The document discusses the concept of diversity in software testing. It provides examples of how diversity has been applied in various testing applications, including test case prioritization and minimization, mutation analysis, and explaining errors in deep neural networks. The key aspects of diversity discussed are the representation of test cases, measures of distance or similarity between cases, and techniques for maximizing diversity. The document emphasizes that the best approach depends on factors like information access, execution costs, and the specific application context.
Applications of Search-based Software Testing to Trustworthy Artificial Intel...
Applications of Search-based Software Testing to Trustworthy Artificial Intel...Lionel Briand This document discusses search-based approaches for testing artificial intelligence systems. It covers testing at different levels, from model-level testing of individual machine learning components to system-level testing of AI-enabled systems. At the model level, search-based techniques are used to generate test inputs that target weaknesses in deep learning models. At the system level, simulations and reinforcement learning are used to test AI components integrated into complex systems. The document outlines many open challenges in AI testing and argues that search-based approaches are well-suited to address challenges due to the complex, non-linear behaviors of AI systems.
Autonomous Systems: How to Address the Dilemma between Autonomy and Safety
Autonomous Systems: How to Address the Dilemma between Autonomy and SafetyLionel Briand Autonomous systems present safety challenges due to their complexity and use of machine learning. Two key approaches are needed to address these challenges: (1) design-time assurance cases to validate safety requirements and (2) run-time monitoring architectures to detect unsafe behavior. Automated testing techniques leveraging metaheuristics and machine learning can help provide evidence for assurance cases and learn conditions to guide run-time monitoring. However, more industrial experience is still needed to properly validate these approaches at scale for autonomous systems.
Mathematicians, Social Scientists, or Engineers? The Split Minds of Software ...
Mathematicians, Social Scientists, or Engineers? The Split Minds of Software ...Lionel Briand This document discusses the split identities of software engineering researchers between being mathematicians, social scientists, or engineers. It notes there are three main communities - formal methods and guarantees, human and social studies, and engineering automated solutions - that have different backgrounds, languages, and research methods. While diversity is good, the communities need to be better connected to work together to solve problems. The document calls for more demand-driven, collaborative research with industry to have a greater impact and produce practical solutions.
Ad
Recently uploaded (20)
Enable Your Cloud Journey With Microsoft Trusted Partner | IFI Tech
Enable Your Cloud Journey With Microsoft Trusted Partner | IFI TechIFI Techsolutions Start your cloud journey with IFI Tech—Microsoft’s trusted partner delivering secure, scalable Azure solutions tailored for your business.
Simplify Task, Team, and Project Management with Orangescrum Work
Simplify Task, Team, and Project Management with Orangescrum WorkOrangescrum Streamline project workflows, team collaboration, and time tracking with orangescrum work, your all-in-one tool for smarter and faster project execution.
Canva Pro Crack Free Download 2025-FREE LATEST
Canva Pro Crack Free Download 2025-FREE LATESTgrete1122g 🌍📱👉COPY & PASTE LINK >> https://click4pc.com/after-verification-click-go-to-download-page/
Canva Pro PC Crack is a practical design app to create beautiful montages and compositions with a lot of resources on the platform.
University Campus Navigation for All - Peak of Data & AI
University Campus Navigation for All - Peak of Data & AISafe Software Navigating around the 1000 acres of UBC Vancouver’s campus may look like a walk in a park, but when you’re a student in a wheelchair trying to find a powered door to the class you’re late for, it is not all that scenic. UBC’s GIS team will share the journey from the original 2003 static map application to the current web application built using FME and the ArcGIS JavaScript API. There were two main goals with this new application: 1. Enable the UBC community to easily interact with and navigate the campus by walking, cycling and accessible means (which include mobility constraints such as steep slopes, stairs, and doors that are not powered). 2. Provide high quality geospatial data that is authoritative and updated using FME to reflect existing conditions on campus including road, sidewalk or building closures for construction. We will cover the unique challenges in creating a navigation application which offers features that Google Maps cannot at the scale of a University campus. Door to door navigation, avoiding stairs and steep slopes, navigating to accessible entrances and up to date construction barriers are some of the features we will discuss.
arctitecture application system design os dsa
arctitecture application system design os dsaza241967 wow amazing see and enjoy for this the best pdf slideshow you could see random shit is what i do best
Humans vs AI Call Agents - Qcall.ai's Special Report
Humans vs AI Call Agents - Qcall.ai's Special ReportUdit Goenka Humans vs Agentic AI Cost Comparison: Cut 85% Support Costs
TL;DR: Agentic AI crushes human costs in L1/L2 support with 85% savings, 300% efficiency gains, and 90%+ success rates.
Your CFO just asked: "Why spend $2.3M annually on L1 support when competitors automated 80% for under $200K?" The humans vs agentic AI cost debate isn't theoretical - it's happening now.
Real Cost Breakdown:
- India L1/L2 Agent: ₹6,64,379 ($8,000) annually (full cost)
- US L1/L2 Agent: $70,000 annually (loaded cost)
- Qcall.ai Agentic AI: ₹6/min ($0.07/min) = 85% savings
Game-Changing Results:
- 47 human agents → 8 agents + AI system
- Operating costs: $2.8M → $420K (85% reduction)
- Customer satisfaction: 78% → 91%
- First call resolution: 72% → 94%
- Languages: 2-3 → 15+ fluently
Why Agentic AI Wins:
- Human Limitations: 8hrs/day, 1-2 languages, 30-45% attrition, mood-dependent, 6-12 weeks training
- Qcall.ai Advantages: 24/7 availability, 97% humanized voice, <60 second response, intelligent qualification, seamless handoff
Industry Impact:
- E-commerce: 82% cost reduction
- Healthcare: 90% appointment automation
- SaaS: 23% satisfaction improvement
- Financial: 95% inquiry automation
Implementation:
- Setup: 2-5 days vs 6-12 weeks hiring
- ROI: Break-even in 3 months
- Scaling: Instant global expansion
Competitive Gap:
- Early Adopters: 85% cost advantage, superior metrics, faster expansion
Late Adopters: Higher costs, limited scalability, compounding disadvantage
Similar automation patterns emerge across industries.
Marketing teams use autoposting.ai for social media automation - same human-AI collaboration model driving L1/L2 transformation.
"We reduced support costs by 78% while improving satisfaction by 31%. ROI was immediate and keeps compounding." - COO, Global SaaS Company
Getting Started:
Calculate current L1/L2 costs
Assess repetitive tasks
Request Qcall.ai demo
Plan 20% pilot
Bottom Line: Companies avoiding this shift hemorrhage millions while competitors scale globally with fraction of costs. The question isn't whether agentic AI will replace routine tasks - it's whether you'll implement before competitors gain insurmountable advantage.
Ready to transform operations? This reveals complete cost analysis, implementation strategy, and competitive advantages.
From Code to Commerce, a Backend Java Developer's Galactic Journey into Ecomm...
From Code to Commerce, a Backend Java Developer's Galactic Journey into Ecomm...Jamie Coleman In a galaxy not so far away, a Java developer advocate embarks on an epic quest into the vast universe of e-commerce. Armed with backend languages and the wisdom of microservice architectures, set out with us to learn the ways of the Force. Navigate the asteroid fields of available tools and platforms; tackle the challenge of integrating location-based technologies into open-source projects; placate the Sith Lords by enabling great customer experiences.
Follow me on this journey from humble Java coder to digital marketplace expert. Through tales of triumph and tribulation, gain valuable insights into conquering the e-commerce frontier - such as the different open-source solutions available - and learn how technology can bring balance to the business Force, large or small. May the code be with you.
AI for PV: Development and Governance for a Regulated Industry
AI for PV: Development and Governance for a Regulated IndustryBiologit AI for pharmacovigilance: Development and Governance for a Regulated Industry.
A survey of regulatory guidance for the use of artificial intelligence in pharmacovigilance and best practices for its safe implementation with the Biologit literature platform as case study.
A Guide to Telemedicine Software Development.pdf
A Guide to Telemedicine Software Development.pdfOlivero Bozzelli Learn how telemedicine software is built from the ground up—starting with idea validation, followed by tech selection, feature integration, and deployment.
Know more about this: https://www.yesitlabs.com/a-guide-to-telemedicine-software-development/
Test Case Design Techniques – Practical Examples & Best Practices in Software...
Test Case Design Techniques – Practical Examples & Best Practices in Software...Muhammad Fahad Bashir This presentation was part of the SQA & PM Bootcamp, where I served as a trainer. It focuses on effective test case design techniques, blending theoretical knowledge with practical application, especially useful for manual testers and QA beginners.
Video Lecture Recording : https://www.facebook.com/share/v/1Z44DiXN5v/
What’s Covered:
🧾 Definition and Purpose of Test Case Design
🚀 Importance of Structured Test Design
🧠 Test Case Design Techniques Overview:
✅ Black Box Techniques
✅ White Box Techniques
✅ Experience-Based Techniques
📝 Manual Test Case Design in Detail
🔐 Real-World Example: Login Page Test Case
🧩 Black Box Methods Explained with Examples:
Equivalence Partitioning (EP)
Boundary Value Analysis (BVA)
Difference Between EP & BVA
📊 Decision Table Testing
💡 Experience-Based Testing: Error Guessing, Exploratory Testing
📂 White Box Techniques (Introductory Overview)
➕ Much More to Help You Build Strong Manual Testing Skills
This resource is especially helpful for students, aspiring QA professionals, and junior testers looking to master test design fundamentals with clarity and practical insight.
Foundations of Marketo Engage - Programs, Campaigns & Beyond - June 2025
Foundations of Marketo Engage - Programs, Campaigns & Beyond - June 2025BradBedford3 Join us for an exciting introductory session on, Foundations of Marketo Engage: Programs, Campaigns & Beyond. Ideal for new and early-stage users looking to build confidence and capability in Marketo Engage.
Our speakers for this session will be:
AJ Navarro
Marketing Operations Manager - Sprout Social
Bobby Coppola
Marketing Manager - Blue Yonder
This event will guide you through the essential steps to build strong, scalable Programs, Channels, Tags, Smart Campaigns, and Reporting Basics in Marketo Engage.
Attendees will gain practical knowledge on how to structure and launch marketing initiatives, leverage templates for efficiency, and implement best practices for campaign execution and measurement. You'll leave with actionable insights you can immediately apply to your own Marketo instance to improve organization, automation, and reporting. Marketo Engage.
Learn how to create impactful programs and understand the best practices for seamless campaign management and effective reporting.
Don’t miss out on this opportunity to gain hands-on guidance from experienced practitioners and to connect with peers who are also new to the platform. Secure your spot today and start building your path to becoming a Marketo pro!
Please feel free to invite colleagues who you think would also benefit from this session.
Which Hiring Management Tools Offer the Best ROI?
Which Hiring Management Tools Offer the Best ROI?HireME In today's cost-focused world, companies must assess not just how they hire, but which hiring management tools deliver the best value for money. Learn more in PDF!
On-Device AI: Is It Time to Go All-In, or Do We Still Need the Cloud?
On-Device AI: Is It Time to Go All-In, or Do We Still Need the Cloud?Hassan Abid As mobile hardware becomes more powerful, the promise of running advanced AI directly on-device is closer than ever. With Google’s latest on-device model Gemini Nano, accessible through the new ML Kit GenAI APIs and AI Edge SDK, alongside the open-source Gemma-3n models, developers can now integrate lightweight, multimodal intelligence that works even without an internet connection. But does this mean we no longer need cloud-based AI? This session explores the practical trade-offs between on-device and cloud AI for mobile apps.
OpenChain Webinar - AboutCode - Practical Compliance in One Stack – Licensing...
OpenChain Webinar - AboutCode - Practical Compliance in One Stack – Licensing...Shane Coughlan OpenChain Webinar - AboutCode - Practical Compliance in One Stack – Licensing, Vulnerabilities, and More
Digital Transformation: Automating the Placement of Medical Interns
Digital Transformation: Automating the Placement of Medical InternsSafe Software Discover how the Health Service Executive (HSE) in Ireland has leveraged FME to implement an automated solution for its “Order of Merit” process, which assigns medical interns to their preferred placements based on applications and academic performance. This presentation will showcase how FME validates applicant data, including placement preferences and academic results, ensuring both accuracy and eligibility. It will also explore how FME automates the matching process, efficiently assigning placements to interns and distributing offer letters. Beyond this, FME actively monitors responses and dynamically reallocates placements in line with the order of merit when offers are accepted or declined. The solution also features a self-service portal (FME Flow Gallery App), enabling administrators to manage the entire process seamlessly, make adjustments, generate reports, and maintain audit logs. Additionally, the system upholds strict data security and governance, utilising FME to encrypt data both at rest and in transit.
Top Time Tracking Solutions for Accountants
Top Time Tracking Solutions for Accountantsoliviareed320 This presentation explores the importance of time tracking for accountants and introduces modern solutions that help streamline billable hours, improve productivity, and eliminate manual tracking errors. Learn how time tracking software tailored for accounting professionals can simplify client billing, ensure compliance, and enhance profitability.
From key challenges in time management to top features to look for in a solution, this presentation offers valuable insights and practical tips for solo accountants, finance teams, and accounting firms looking to optimize their time-tracking workflows.https://www.invoicera.com/blog/invoicing/best-time-tracking-software-for-accountants/
Streamlining CI/CD with FME Flow: A Practical Guide
Streamlining CI/CD with FME Flow: A Practical GuideSafe Software Join us as we explore how to deploy a fault-tolerant FME Flow installation using FME Flow’s IaC templates alongside Terraform, AWS, and GitHub in a CI/CD workflow. This session will also cover how to leverage FME’s CI/CD capabilities for FME Flow upgrades and FME object migration between different environments, ensuring seamless automation and scalability.
CodeCleaner: Mitigating Data Contamination for LLM Benchmarking
CodeCleaner: Mitigating Data Contamination for LLM Benchmarkingarabelatso The pdf-version of "CodeCleaner: Mitigating Data Contamination for LLM Benchmarking" presented in Internetware 2025.
Modern Platform Engineering with Choreo - The AI-Native Internal Developer Pl...
Modern Platform Engineering with Choreo - The AI-Native Internal Developer Pl...WSO2 Building and operating internal platforms has become increasingly complex — sprawling toolchains, rising costs, and fragmented developer experiences are slowing teams down. It’s time for a new approach.
This slide deck explores modern platform engineering with Choreo, WSO2’s AI-native Internal Developer Platform as a Service. Discover how you can streamline platform operations, empower your developers, and drive faster innovation — all while reducing complexity and cost.
Learn more: https://wso2.com/choreo/platform-engineering/
Y - Recursion The Hard Way GopherCon EU 2025
Y - Recursion The Hard Way GopherCon EU 2025Eleanor McHugh In most modern programming languages, including Go, it's possible for a function to call itself recursively. It's such a mainstream technique that we've come to take it for granted.
But what if functions couldn't call themselves?
There's a branch of mathematics called Combinatorics which answers just these kinds of questions, and this presentation takes these ideas and phrases them in Go to teach a deeper view of computation through the medium of code.
Test Case Design Techniques – Practical Examples & Best Practices in Software...
Test Case Design Techniques – Practical Examples & Best Practices in Software...Muhammad Fahad Bashir
Search-driven String Constraint Solving for Vulnerability Detection
- 1. .lusoftware verification & validation VVS Julian Thomé, Lwin Khin Shar, Domenico Bianculli and Lionel Briand Search-driven String Constraint Solving for Vulnerability Detection
- 2. Injection vulnerabilities and XSS are serious threats 2
- 3. protected void authenticate() { String user = req.getParameter("user"); // SOURCE String pin = req.getParameter("pin"); // SOURCE String token = req.getParameter("token"); // SOURCE Document doc = db.parse(new File("users.xml")); if(user.isEmpty() || pin.isEmpty() || !token.matches("[0-9]{8}")) { // … } else { String q = "/users/user[@id='" + ESAPI.encoder().encodeForXPath(user) + "' and @pin=" + ESAPI.encoder().encodeForXPath(pin) + "]"; // … NodeList nl=(NodeList)xpath.evaluate(q); // SINK // … } } 3 A vulnerable example program
- 4. protected void authenticate() { String user = req.getParameter("user"); String pin = req.getParameter("pin"); String token = req.getParameter("token"); Document doc = db.parse(new File("users.xml")); if(user.isEmpty() || pin.isEmpty() || !token.matches("[0-9]{8}")) { // … } else { String q = "/users/user[@id='" + ESAPI.encoder().encodeForXPath(user) + "' and @pin=" + ESAPI.encoder().encodeForXPath(pin) + "]"; // … NodeList nl=(NodeList)xpath.evaluate(q); // … } } 4 A vulnerable example program "0 or 1" "eve" The program is vulnerable to XPath Injection attacks "/users/user[@id='evil' and @pin=0 or 1]"
- 6. Vulnerability Analysis: State-of-the-Art Program Path Conditions Threat Model + Attack Conditions Symbolic Execution SAT = vulnerable UNSAT = not vulnerable Constraint Solving 6
- 7. Limitation of Constraint Solvers Only limited support for (complex) string operations provided by the state-of-the-art constraint solvers: 7 - String replacement and/or sanitisation operations - String libraries of programming languages provide hundreds of operations (e.g. Java String library, Apache Commons)
- 8. Workaround 1: Extending Solver Constraint Solvers could be extended in order to support new operations 8 Problems: - Not trivial and requires expert knowledge - Not scalable to the size of a complete string library of a modern programming language
- 9. Workaround 2: Re-expressing Constraints Constraints could be re-expressed in terms of constraints which are natively supported by the constraint solver 9 Problem: Increased complexity of generated constraint, potentially leading to scalability issues
- 10. However, in practice … 10
- 11. 11 Constraint Solvers fail or return an error CVC4 Z3-str2 Remind audience about the limitation of state-of-the-art
- 12. Our Approach: Search-driven String Constraint Solving 12
- 13. Search-driven String Constraint Solving External Solver (CVC4, Z3-str2, …) Attack Condition constraint with unsupported operations solutions of constraint with supported operations SAT/ UNSAT/ TIMEOUT Hybrid Constraint Solving 13
- 14. 14 Hybrid Constraint Solving (ACO-Solver) 1. Automata-based solver solves all constraints it supports and returns a solution for every variable in terms of an FSM 2. Search-based solver searches for paths in the solution automata that make the constraint satisfiable Automata-based solver reduces the search space
- 15. Search-driven String Constraint Solving External Solver (CVC4, Z3-str2, …) Attack Condition constraint with unsupported operations solutions of constraint with supported operations SAT/ UNSAT/ TIMEOUT Automata-based Solver Search-based Solver ACO-Solver 15
- 16. len(v0user ) > 0 len(v0pin ) > 0 v1user = encodeForXPath(v0user ) v0q = concat("/users/user[@id='", v1user ) v1q = concat(v0q , "' and @pin=") v1pin = encodeForXPath(v0pin ) v2q = concat(v1q , v1pin ) v3q = concat(v2q , "]") matches(v1pin , "[0-9]+ [Oo][Rr] 1=1") matches(v0token , "[0-9]{8}") Attack Condition Decomposition 16 len(v0user ) > 0 len(v0pin ) > 0 matches(v0token , "[0-9]{8}") v1user = encodeForXPath(v0user ) v0q = concat("/users/user[@id='", v1user ) v1q = concat(v0q , "' and @pin=") v1pin = encodeForXPath(v0pin ) v2q = concat(v1q , v1pin ) v3q = concat(v2q , "]") matches(v1pin , "[0-9]+ [Oo][Rr] 1=1")
- 17. Attack Condition Decomposition 17 len(v0user ) > 0 len(v0pin ) > 0 v1user = encodeForXPath(v0user ) v0q = concat("/users/user[@id='", v1user ) v1q = concat(v0q , "' and @pin=") v1pin = encodeForXPath(v0pin ) v2q = concat(v1q , v1pin ) v3q = concat(v2q , "]") matches(v1pin , "[0-9]+ [Oo][Rr] 1=1") len(v0user ) > 0 len(v0pin ) > 0 matches(v0token , "[0-9]{8}") v1user = encodeForXPath(v0user ) v0q = concat("/users/user[@id='", v1user ) v1q = concat(v0q , "' and @pin=") v1pin = encodeForXPath(v0pin ) v2q = concat(v1q , v1pin ) v3q = concat(v2q , "]") matches(v1pin , "[0-9]+ [Oo][Rr] 1=1") matches(v0token , "[0-9]{8}")
- 18. Provide every attack condition partition as input to the external solver Search-driven String Constraint Solving 18 External Solver (CVC4, Z3-str2, …) Attack Condition constraint with unsupported operations solutions of constraint with supported operations SAT/ UNSAT/ TIMEOUT Automata-based Solver Search-based Solver ACO-Solver
- 19. 19 len(v0user ) > 0 len(v0pin ) > 0 v1user = encodeForXPath(v0user ) v0q = concat("/users/user[@id='", v1user ) v1q = concat(v0q , "' and @pin=") v1pin = encodeForXPath(v0pin ) v2q = concat(v1q , v1pin ) v3q = concat(v2q , "]") matches(v1pin , "[0-9]+ [Oo][Rr] 1=1") matches(v0token , "[0-9]{8}") SAT/UNSAT/Crash Attack Condition Partition SAT/UNSAT/Crash Invoke External Solver Result
- 20. Invoke External Solver 20 matches(v0token , "[0-9]{8}") SAT ResultAttack Condition Partition Crash len(v0user ) > 0 len(v0pin ) > 0 v1user = encodeForXPath(v0user ) v0q = concat("/users/user[@id='", v1user ) v1q = concat(v0q , "' and @pin=") v1pin = encodeForXPath(v0pin ) v2q = concat(v1q , v1pin ) v3q = concat(v2q , "]") matches(v1pin , "[0-9]+ [Oo][Rr] 1=1")
- 21. All the Attack Condition partitions with unsupported operations are solved by ACO-Solver Search-driven String Constraint Solving 21 External Solver (CVC4, Z3-str2, …) Attack Condition constraint with unsupported operations solutions of constraint with supported operations SAT/ UNSAT/ TIMEOUT Automata-based Solver Search-based Solver ACO-Solver
- 22. - An unsupported operation (foo) has to be invokable and its output out has to be observable - Search a set of inputs that generate an output (out) which satisfies all the constraint which are imposed on it Search-based Solving 22 out=foo(i0 … in)
- 23. Ant Colony Optimisation (ACO) - Suited for graph searching problems - Stochastic approach in nature, which allows for escaping from local optima - Inherent parallelism - Inspired by the behaviour of ants (leaving pheromone traces on paths leading to food) 23
- 24. Fitness Function 24 - Assess the quality of a potential solution - A lower fitness implies a higher quality of the solution - Different fitness functions for 1. Numeric constraints (Korel) 2. String constraints (Levenshtein) 3. Regular expressions (Myers and Miller)
- 25. ACO Algorithm 1 Construction of solution 1,1 Build set of solution components 1,2 Determine fitness of solution components 1,3 Selecting the best solution components 2 Application of local search 3 Update of pheromone values 25
- 26. Evaluation 26
- 27. Benchmark and Evaluation Settings 27 - 43 web programs from 9 Java Web applications/services (1 KLOC - 52 KLOC) - Attack conditions for 64 vulnerable and 40 non- vulnerable paths with various vulnerability types (SQLi, XMLi, Xpathi, LDAPi, XSS) - The timeout for solving each attack condition was set to 30s
- 28. RQ1: Benefit How does the proposed approach improve the effectiveness of state-of- the-art solvers for solving constraints related to vulnerability detection? 28
- 29. Z3-str2 Z3-str2 + ACO-Solver ✔ vuln. detected ✔ ∆ vuln. detected 19 3 4,7 % 65 46 46 71,9 % ACO-Solver significantly improves the recall (# detected vulnerabilities) of Z3-str2/CVC4 RQ1: Benefit CVC4 CVC4 + ACO-Solver ✔ vuln. detected ✔ ∆ vuln. detected 72 55 85,9 % 83 11 64 100 % 29 explain what the limitations of Z3-str2 are - Z3-str has some limitations when it comes to sym
- 30. RQ2: Cost Is the cost of using our technique affordable in practice? 30
- 31. 31 The cost of using our technique is affordable, because - we can detect significantly more vulnerabilities - vulnerability detection is an offline activity Z3-str2 Z3-str2 + ACO- Solver CVC4 CVC4 + ACO- Solver time (s) 100,28 1.518,33 4,96 728,57 RQ2: Cost
- 32. RQ3: Role of the Automata-based solver Does the automata-based solver contribute to the effectiveness of the search-based procedure? 32
- 33. 33 The automata-based solver plays a fundamental role in achieving a higher effectiveness RQ3: Role of the Automata-based solver Z3-str2 Z3-str2 + modACO-Solver ✔ vuln. detected t(s) ✔ ∆ vuln. detected t(s) 19 3 4,7 % 100,28 19 0 3 4,7 % 2.651,66 CVC4 CVC4 + modACO-Solver ✔ vuln. detected t(s) ✔ ∆ vuln. detected t(s) 72 55 85,9 % 4,69 73 1 56 87,5 % 927,75
- 34. 34 Conclusion Additional Information: https://github.com/julianthome/acosolver
- 35. Making constraint solving for vulnerability detection practical 35 Additional Information: https://github.com/julianthome/acosolver