Operations Research, Combinatorics and Optimisation (ORCO) 2nd year

This course presents advanced methods and technics for Operations Research.

Reminder :

Linear Programming, Dynamic Programming, MIP modelling and BB
Complexity (P, NP, Co-NP)

Advanced MIP :

formulation, cuts, bounds
applications
lagragian relaxation
column generation
Benders decomposition
Solvers

Constraint Programming

Heuristics

local search
approximation algorithms

Visit the full page about this course

The aim of this course is to provide a broad knowledge of fundamental problems in Combinatorial Optimization to show their algorithmic solutions and to derive min-max results on them. In order to achieve this goal a new object called a polyhedron is introduced. This polyhedral approach helps to shed new light on some classic results of Combinatorial Optimization.

Syllabus: Study of polyhedra associated to problems of Combinatorial Optimization ; Theory of blocking polyhedra ; Connectivity: shortest paths, spanning trees and spanning arborescences of minimum weight ; Flows: Edmonds-Karp Algorithm, Goldberg-Tarjan Algorithm, minimum cost flows ; Matchings: Hungarian method, Edmonds' Algorithm, Chinese postman problem; Matroids: greedy algorithm, intersection of two matroids ; Graph coloring ; Applications coming from various areas of Operations Research.

Visit the full page about this course

The objective of this course is to present different techniques to handle uncertainty in decision problems. These techniques will be illustrated on several applications e.g. inventory control, scheduling, energy, machine learning.

Syllabus : Introduction to uncertainty in optimization problems; Reminders (probability, dynamic programming, ...); Markov chains; Markov decision processes; Stochastic programming; Robust optimization

Visit the full page about this course

Visit the full page about this course

The aim of this course is to learn how to use the structure of graphs and other discrete objects to obtain general results on them, and in particular efficient algorithms solving important problems.

We will cover the following topics:

• Structural Graph Theory: we will study the structure of important graph classes with nice algorithmic properties (planar graphs, interval graphs, comparability graphs, ...) and show several concrete problems that can be solved thanks to their structural properties.
• Graph Drawing: with the rise of Big Data, representing huge data sets is a fundamental challenge. Efficient ways to represent large graphs will be presented.
• Codes: we will see various codes (dominating, locating, identifying, ...) in graphs and their applications.
• Extremal combinatorics: the typical question in this field is "what global condition do we need to impose in some graph in order to make sure that some nice structure appears locally?" We will introduce a powerful tool called "the probabilistic method", an show how it can be applied to solve problems in this important area of research.

Visit the full page about this course

Visit the full page about this course

Visit the full page about this course

This course offers the possibility for the students to gain some experience by facing open/difficult combinatorial problems.

The goal is to model and solve a combinatorial problem with direct industrial applications. We expect the students to take a variety of approaches (local search, compact/extended linear programming formulations, constraint programming, ...) and establish useful results (lower bounds, cuts, complexity,...).

The experimental results will be compared to the litterature (a known academic open benchmark will be available in this case) or will be validated by the industrial partner.

Visit the full page about this course

Visit the full page about this course

A central ingredient in many methods for symbolic and concolic executions, for finding inductive invariants, and for checking their inductiveness, is _satisfiability modulo theory_ (SMT). SMT is based on Boolean satisfiability testing (SAT), that is, checking whether a propositional formula has a solution. There exist algorithms for solving SAT and many cases of SMT, but the issue here is computational complexity: SAT is a hard problem.

Visit the full page about this course

Today, parallel computing is omnipresent across a large spectrum of computing platforms, from processor cores or GPUs up to Supercomputers and Cloud platforms. The largest Supercomputers gather millions of processing units and are heading towards Exascale (a quintillion or 10^18 flops - http://top500.org). If parallel processing was initially targeting scientific computing, it is now part of many domains like Big Data analytics and Deep Learning. But making an efficient use of these parallel resources requires a deep understanding of architectures, systems, parallel programming models, and parallel algorithms.

Overview:

The class is organized around weekly lectures, discussions and help time. The presented materials that will be available each week on the class web page. To get practical experience and good understanding of the main concepts the students are expected to develop short programs and experiments. Students will also have to prepare a presentation or a written report on research articles. Students will have access to Grid'5000 parallel machines and the SimGrid simulator for experiments.

This class is organized around 2 main blocks:

Overview of parallel systems:

1. • Introduction to parallelism from GPU to supercomputers.
   • Hardware and system considerations for parallel processing (multi-core architectures, process and thread handling, cache efficiency, remote data access, atomic instructions)
   • Parallel programming: message passing, one-sided communications, task-based programming, work stealing based runtimes (MPI, Cilk, TBB, OpenMP).
   • Modeling of parallel programs and platforms. Locality, granularity, memory space, communications.
   • Parallel algorithms, collective communications, topology aware algorithms.
   • Scheduling: list algorithms; partitioning techniques. Application to resource management (PBS, LSF, SGE, OAR).
   • Large scale scientific computing: parallelization of Lagrangian and Eulerian solvers, parallel data analytics and scientific visualization.
   • AI and HPC: how parallelism is used at different levels to accelerate machine learning and deep learning using supercomputers.

Functional parallel programming:

We propose to study a clean and modern approach to the design of parallel algorithms: functional programming. Functional languages are known to provide a different and cleaner programming experience by allowing a shift of focus from "how to do" on "what to do".
If you take for example a simple dot product between two vectors. In c language you might end up with:

unsigned int n = length(v1); double s = 0.0; for (unsigned int i = 0 ; i < n ; i++) { s += v1[i] * v2[i]; }

In python however you could write:

return sum(e1*e2 for e1, e2 in zip(v1, v2))

You can easily notice that the c language code displayed here is highly sequential with a data-flow dependence on the i variable. It intrinsically contains an ordering of operations because it tells you how to do things to obtain the final sum. On the other end the python code exhibits no dependencies at all. It does not tell you how to compute the sum but just what to compute: the sum of all products.

In this course we will study how to express parallel operations in a safe and performant way. The main point is to study parallel iterators and their uses but we will also consider classical parallel programming schemes like divide and conquer. We will both study the theoretical complexity of different parallel algorithms and practice programming and performance analysis on real machines.
All applications will be developed in the RUST programming language around the Rayon parallel programming library.
No previous knowledge of the rust language is required as we will introduce it gradually during the course. You need however to be proficient in at least one low level language (typically C or C++)

Visit the full page about this course

Multi-agent systems (MAS) is a very active field of AI research, with multiple industrial and societal applications. The 2 main fields of application concern Distributed Problem Solving (DPS) and Agent-Based Modelling and Simulation (ABMS). The goal of this course is to understand the concepts of agents, multi-agent systems, models and simulations, and to learn how to design such models. 

This course introduces the field of MAS, various theoretical aspects (agent architectures, reasoning, interactions, game theory, social choice, etc), as well as practical applications from recent research. The focus is mostly on agent-based social simulation, and how to integrate psychological aspects in agents (so-called “human factors”: emotions, biases…) to make them more human-like and realistic. Applications discussed include epidemics modelling, computational economy, crisis management, urban planning, serious games, etc. The practical part of the course comprises several tutorials with various agent-based modelling platforms (in particular GAMA and Netlogo), scientific papers discussions, and analysis and/or extension of existing models. 

Visit the full page about this course

Modern computing increasingly takes advantage of large amounts of distributed data and knowledge. This is grounded on theoretical principles borrowing to several fields of computer science such as programming languages, data bases, structured documentation, logic and artificial intelligence. The goal of this course is to present some of them, the problems that they solve and those that they uncover. The course considers two perspectives on data and knowledge: interpretation (what they mean), analysis (what they reveal) and processing (how can they be traversed efficiently and transformed safely).

The course offers a semantic perspective on distributed knowledge. Distributed knowledge may come from data sources using different ontologies on the semantic web, autonomous software agents learning knowledge or social robots interacting with different interlocutors. The course adopts a synthetic view on these. It first presents principles of the semantics of knowledge representation (RDF, OWL). Ontology alignments are then introduced to reduce the heterogeneity between distributed knowledge and their exploitation for answering federated queries is presented. A practical way for cooperating agents to evolve their knowledge is cultural knowledge evolution that is then illustrated. Finally, the course defines dynamic epistemic logics as a way to model the communication of knowledge and beliefs.

The course also introduces a perspective on programming language foundations, algorithms and tools for processing structured information, and in particular tree-shaped data. It consists in an introduction to relevant theoretical tools with an application to NoSQL (not only SQL) and XML technologies in particular. Theories and algorithmic toolboxes such as fundamentals of tree automata and tree logics are introduced, with applications to practical problems found for extracting information. Applications include efficient query evaluation, memory-efficient validation of document streams, robust type-safe processing of documents, static analysis of expressive queries, and static type-checking of programs manipulating structured information. The course also aims at presenting challenges, important results, and open issues in the area.

Visit the full page about this course

The course aims to provide the fundamental basis for a sound scientific methodology of experimental evaluation in computer science. This lecture emphasizes on methodological aspects of measurement and on the statistics needed to analyze computer systems, human-computer interaction systems, and machine learning systems. We first sensibilize the audience to reproducibility issues related to empirical research in computer science as well as to ethical and scientific integrity aspects. Then we present tools that help address the aforementioned issues and we give the audience the basis of probabilities and statistics required to develop sound experiment designs. The content of the lecture is therefore both theoretical and practical, illustrated by a lot of case studies and practical sessions. The goal is not to provide analysis recipes or techniques that researchers can blindly apply but to make students develop critical thinking and understand some simple (and possibly not-so-simple) tools so that they can both readily use and explore later on.

Visit the full page about this course

The course is divided in two complementary parts: distributed systems and data management.

Part 1: Distributed systems

Summary

Distributed systems are omnipresent. They are formed by a set of computing devices, interconnected by a network, that collaborate to perform a task. Distributed systems execute on a wide range of infrastructures: from Cloud datacenters to wireless sensor networks.

The goal of this course is to study the main algorithms used at the core of Distributed systems. These algorithms must be efficient and robust. An algorithm is efficient when it sustains a high level of performance. Performance can be measured using various metrics such as throughput, latency, response time. An algorithm is robust when it is able to operate despite the occurrence of various types of (network and/or machine) and attacks.

Content

During the course, we will cover several topics that are listed below:

• Event-driven formalisms for distributed algorithms
• Basic abstractions: processes, links
• Failure detector algorithms
• Leader election algorithms
• Broadcasting algorithms
• Distributed shared memory algorithms
• Consensus algorithms
• Epidemic algorithms
• Performance models for distributed systems

Part 2: Data management

Summary

The ability to process large amounts of data is key to both industry and research today. As computing systems are getting larger, they generate more data that need to be analyzed to extract knowledge.

Data management infrastructures are growing fast, leading to the creation of large data centers and federations of data centers. Suitable software infrastructures should be used to store and process data in this context. Big Data software systems are built to take advantage of a large set of distributed resources to efficiently process massive amounts of data while being able to cope with failures that are frequent at such a scale.

In addition to the amount of data to be processed, the other main challenge that such Big Data systems need to deal with is time. For some use cases, the earlier the results of a data analysis is obtained, the more valuable the result is. Some Big Data systems especially target stream processing where data are processed in real time.

Through lectures and practical sessions, this course provides an overview of the software systems that are used to store and process data at large scale. The following topics will be covered:

• Map-Reduce programming model
• In-memory data processing
• Stream processing (data movement and processing)
• Large scale distributed data storage (distributed file systems, NoSQL databases)

Throughout the lectures, the challenges associated with performance and fault tolerance will also be discussed.

Visit the full page about this course

The course present the main cryptographic primitives and security protocols, focusing on security parameters and properties.

Pedagogical goals:

- generic cryptographic primitives: one-way, trap-door and hash functions; random generators; symmetric and assymertic cipher; interactive protocols;

- security properties : complexity and reduction proofs; undistinguidhability; non-malleability; soundness, completeness and zero-knowledge; confidentiality; authentication; privacy; non-repudiation

- use, deployment and integration of protocols in standard crypro lib (eg open-ssl)

- security proofs : fundations and verufucation based on tools (eg avispa)

Visit the full page about this course

Statistical learning is about the construction and study of systems that can automatically learn from data. With the emergence of massive datasets commonly encountered today, the need for powerful machine learning is of acute importance. Examples of successful applications include effective web search, anti-spam software, computer vision, robotics, practical speech recognition, and a deeper understanding of the human genome. This course gives an introduction to this exciting field. In the first part, we will introduce basic techniques such as logistic regression, multilayer perceptrons, nearest neighbor approaches, both from a theoretical and methodological point of views. In the second part, we will focus on more advanced techniques such as kernel methods, which is a versatile tool to represent data, in combination with (un)supervised learning techniques that are agnostic to the type of data that is learned from. The learning techniques that will be covered include regression, classification, clustering and dimension reduction. We will cover both the theoretical underpinnings of kernels, as well as a series of kernels that are important in practical applications. Finally we will touch upon topics of active research, such as large-scale kernel methods and the use of kernel methods to develop theoretical foundations of deep learning models.

Visit the full page about this course

Machine Learning is one of the key areas of Artificial Intelligence and it concerns the study and the development of quantitative models that enables a computer to perform tasks without being explicitly programmed to do them. Learning in this context is hence to recognize complex forms and to make intelligent decisions. Given all existing entries, the difficulty of this task lies in the fact that all possible decisions is usually very complex to enumerate. To get around that, machine learning algorithms are designed in order to gain knowledge on the problem to be addressed based on a limited set of observed data extracted from this problem. To illustrate this principle, consider the supervised learning task, where the prediction function, which infers a predicted output for a given input, is learned over a finite set of labeled training examples, where each instance of this set is a pair constituted of a vector characterizing an observation in a given vector space, and an associated desired response for that instance (also called desired output). After the training step, the function returned by the algorithm is sought to give predictions on new examples, which have not been used in the learning process, with the lowest probability of error. The underlying assumption in this case is that the examples are, in general, representative of the prediction problem on which the function will be applied. We expect that the learning algorithm produces a function that will have a good generalization performance and not the one that is able to perfectly reproduce the outputs associated to the training examples. Guarantees of learnability of this process were studied in the theory of machine learning largely initiated by Vladimir Vapnik. These guarantees are dependent on the size of the training set and the complexity of the class of functions where the algorithm searches for the prediction function. Emerging technologies, particularly those related to the development of Internet, reshaped the domain of machine learning with new learning frameworks that have been studied to better tackle the related problems. One of these frameworks concerns the problem of learning with partially labeled data, or semi-supervised learning, which development is motivated by the effort that has to be made to construct labeled training sets for some problems, while large amount of unlabeled data can be gathered easily for these problems. The inherent assumption, in this case, is that unlabeled data contain relevant information about the task that has to be solved, and that it is a natural idea to try to extract this information so as to provide the learning algorithm more evidence. From these facts were born a number of works that intended to use a small amount of labeled data simultaneously with a large amount of unlabeled data to learn a prediction function.

The intent of this course is to propose a broad introduction to the field of Machine Learning, including discussions of each of the major frameworks, supervised, unsupervised, semi-supervised and reinforcement learning.

Visit the full page about this course

Causality is at the core of our vision of the world and of the way we reason. It has long been recognized as an important concept and was already mentioned in the ancient Hindu scriptures: “Cause is the effect concealed, effect is the cause revealed”. Even Democritus famously proclaimed that he would rather discover a causal relation than be the king of presumably the wealthiest empire of his time. Nowadays, causality is seen as an ideal way to explain observed phenomena and to provide tools to reason on possible outcomes of interventions and what-if experiments, which are central to counterfactual reasoning, as ‘‘what if this patient had been given this particular treatment?’’

Visit the full page about this course

This course is related to mathematical and statistical methods which are very used in supervised learning.

It contains two parts.

In the first part, we will focus on parametric modeling. Starting with the classical linear regression, we will describe several families of estimators that work when considering high-dimensional data, where the classical least square estimator does not work. Model selection and model assessment will particularly be described.

In the second part, we shall focus on nonparametric methods. We will present several tools and ingredients to predict the future value of a variable. We shall focus on methods for non parametric regression from independent to correlated training dataset. We shall also study some methods to avoid the overfitting in supervised learning.

This course will be followed by practical sessions with the R software.

Visit the full page about this course

The subject of this half-semester course are more advanced methods in convex optimization. It consists of 6 lectures, 2 x 1,5 hours each, and can be seen as continuation of the course “Non-smooth methods in convex optimization”.

This course deals with:

Evaluation : A two-hours written exam (E1) in December. For those who do not pass there will be another two-hours exam (E2) in session 2 in spring.

Visit the full page about this course

Visit the full page about this course

Visit the full page about this course

This course presents advanced methods and technics for Operations Research.

Reminder :

Linear Programming, Dynamic Programming, MIP modelling and BB
Complexity (P, NP, Co-NP)

Advanced MIP :

formulation, cuts, bounds
applications
lagragian relaxation
column generation
Benders decomposition
Solvers

Constraint Programming

Heuristics

local search
approximation algorithms

Visit the full page about this course

The aim of this course is to provide a broad knowledge of fundamental problems in Combinatorial Optimization to show their algorithmic solutions and to derive min-max results on them. In order to achieve this goal a new object called a polyhedron is introduced. This polyhedral approach helps to shed new light on some classic results of Combinatorial Optimization.

Syllabus: Study of polyhedra associated to problems of Combinatorial Optimization ; Theory of blocking polyhedra ; Connectivity: shortest paths, spanning trees and spanning arborescences of minimum weight ; Flows: Edmonds-Karp Algorithm, Goldberg-Tarjan Algorithm, minimum cost flows ; Matchings: Hungarian method, Edmonds' Algorithm, Chinese postman problem; Matroids: greedy algorithm, intersection of two matroids ; Graph coloring ; Applications coming from various areas of Operations Research.

Visit the full page about this course

The objective of this course is to present different techniques to handle uncertainty in decision problems. These techniques will be illustrated on several applications e.g. inventory control, scheduling, energy, machine learning.

Syllabus : Introduction to uncertainty in optimization problems; Reminders (probability, dynamic programming, ...); Markov chains; Markov decision processes; Stochastic programming; Robust optimization

Visit the full page about this course

Visit the full page about this course

The aim of this course is to learn how to use the structure of graphs and other discrete objects to obtain general results on them, and in particular efficient algorithms solving important problems.

We will cover the following topics:

• Structural Graph Theory: we will study the structure of important graph classes with nice algorithmic properties (planar graphs, interval graphs, comparability graphs, ...) and show several concrete problems that can be solved thanks to their structural properties.
• Graph Drawing: with the rise of Big Data, representing huge data sets is a fundamental challenge. Efficient ways to represent large graphs will be presented.
• Codes: we will see various codes (dominating, locating, identifying, ...) in graphs and their applications.
• Extremal combinatorics: the typical question in this field is "what global condition do we need to impose in some graph in order to make sure that some nice structure appears locally?" We will introduce a powerful tool called "the probabilistic method", an show how it can be applied to solve problems in this important area of research.

Visit the full page about this course

Visit the full page about this course

Visit the full page about this course

This course offers the possibility for the students to gain some experience by facing open/difficult combinatorial problems.

The goal is to model and solve a combinatorial problem with direct industrial applications. We expect the students to take a variety of approaches (local search, compact/extended linear programming formulations, constraint programming, ...) and establish useful results (lower bounds, cuts, complexity,...).

The experimental results will be compared to the litterature (a known academic open benchmark will be available in this case) or will be validated by the industrial partner.

Visit the full page about this course

Visit the full page about this course

Visit the full page about this course