Résumé

Supervised machine learning techniques include classification and regression. In regression, the objective is to map a real-valued output to a set of input features. The main challenge that existing methods for regression encounter is how to maintain an accuracy-simplicity balance. Since Regression Trees (RTs) are simple to interpret, many existing works have focused on proposing RT and Model Tree (MT) induction algorithms. MTs are RTs with a linear function at the leaf nodes rather than a numerical value are able to describe the relationship between the inputs and the output. Traditional RT induction algorithms are based on a top-down strategy which often leads to a local optimal solution. Other global approaches based on Evolutionary Algorithms (EAs) have been proposed to induce RTs but they can require an important calculation time which may affect the convergence of the algorithm to the solution. In this paper, we introduce a novel approach called Bi-level Evolutionary Model Tree Induction algorithm for regression, that we call BEMTI, and which is able to induce an MT in a bi-level design using an EA. The upper-level evolves a set of MTs using genetic operators while the lower-level optimizes the Linear Models (LMs) at the leaf nodes of each MT in order to fairly and precisely compute their fitness and obtain the optimal MT. The experimental study confirms the outperformance of our BEMTI compared to six existing tree induction algorithms on nineteen datasets.

Résumé

This paper presents a new evolutionary method and tool called BMLDS (Bi-level Multi-Label Detection of Smells) that optimizes a population of classifier chains for the multi-label detection of smells. As the chain is sensitive to the labels' (i.e., smell types) order, the chains induction task is framed as a bi-level optimization problem, where the upper-level role is to search for the optimal order of each considered chain while the lower-level one is to generate the chains. This allows taking into consideration the interactions between smells in the multi-label detection process. The statistical analysis of the experimental results reveals the merits of our proposal with respect to several existing works.

Résumé

Code smells, also known as anti-patterns, are poor design or implementation choices that hinder program comprehensibility and maintainability. While several code smell detection methods have been proposed, Mantyla et al. identified the uncertainty issue as one of the major individual human factors that may affect developer’s decisions about the smelliness of software classes: they may indeed have different opinions mainly due to their different knowledge and expertise. Unfortunately, almost all the existing approaches assume data perfection and neglect the uncertainty when identifying the labels of the software classes. Ignoring or rejecting any uncertainty form could lead to a considerable loss of information, which could significantly deteriorate the effectiveness of the detection and identification processes. Inspired by our previous works and motivated by the interesting performance of the PDT (Possibilistic Decision Tree) in classifying uncertain data, we propose ADIPE (Anti-pattern Detection and Identification using Possibilistic decision tree Evolution), as a new tool that evolves and optimizes a set of detectors (PDTs) that could effectively deal with software class labels uncertainty using some concepts from the Possibility theory. ADIPE uses a PBE (Possibilistic Base of Examples: a dataset with possibilistic labels) that it is built using a set of opinion-based classifiers (i.e., a set of probabilistic classifiers) with the aim to simulate human developers’ uncertainty. A set of advisors and probabilistic classifiers are employed in order to mimic the subjectivity and the doubtfulness of software engineers. A detailed experimental study is conducted to show the merits and outperformance of ADIPE in dealing with uncertainty in code smells detection and identification with respect to four relevant state-of-the-art methods, including the baseline PDT. The experimental study was performed in uncertain and certain environments based on two suitable metrics: PF-measure_dist (Possibilistic F-measure_Distance) and IAC (Information Affinity Criterion); which corresponds to the F-measure and Accuracy (PCC) for the certain case. The obtained results for the uncertain environment reveal that for the detection process, the PF-measure_dist of ADIPE ranges within [0.9047 and 0.9285], and its IAC lies within [0.9288 and 0.9557]; while for the identification process, the PF-measure_dist of ADIPE is in [0.8545, 0.9228], and its IAC lies within [0.8751, 0.933]. ADIPE is able to find 35% more code smells with uncertain data than the second best algorithm (i.e., BLOP). In addition, ADIPE succeeds to decrease the number of false alarms (i.e., misclassified smelly instances) with a rate equals to 12%. Our proposed approach is also able to identify 43% more smell types than BLOP and decreases the number of false alarms with a rate equals to 32%. Similar results were obtained for the certain environment, which demonstrate the ability of ADIPE to also deal with the certain environment.

Résumé

Résumé

Cost-sensitive learning is one of the most adopted approaches to deal with data imbalance in classification. Unfortunately, the manual definition of misclassification costs is still a very complicated task, especially with the lack of domain knowledge. To deal with the issue of costs' uncertainty, some researchers proposed the use of intervals instead of scalar values. This way, each cost would be delimited by two bounds. Nevertheless, the definition of these bounds remains as a very complicated and challenging task. Recently, some researches proposed the use of genetic programming to simultaneously build classification trees and search for optimal costs' bounds. As for any classification tree there is a whole search space of costs' bounds, we propose in this paper a bi-level evolutionary approach for interval-based cost-sensitive classification tree induction where the trees are constructed at the upper level while misclassification costs intervals bounds are optimized at the lower level. This ensures not only a precise evaluation of each tree but also an effective approximation of optimal costs intervals bounds. The performance and merits of our proposal are shown through a detailed comparative experimental study on commonly used imbalanced benchmark data sets with respect to several existing works.

Résumé

Discretization-based feature selection (DBFS) approaches have shown interesting results when using several metaheuristic algorithms, such as particle swarm optimization (PSO), genetic algorithm (GA), ant colony optimization (ACO), etc. However, these methods share the same shortcoming which consists in encoding the problem solution as a sequence of cut-points. From this cut-points vector, the decision of deleting or selecting any feature is induced. Indeed, the number of generated cut-points varies from one feature to another. Thus, the higher the number of cut-points, the higher the probability of selecting the considered feature; and vice versa. This fact leads to the deletion of possibly important features having a single or a low number of cut-points, such as the infection rate, the glycemia level, and the blood pressure. In order to solve the issue of the dependency relation between the feature selection (or removal) event and the number of its generated potential cut-points, we propose to model the DBFS task as a bilevel optimization problem and then solve it using an improved version of an existing co-evolutionary algorithm, named I-CEMBA. The latter ensures the variation of the number of features during the migration process in order to deal with the multimodality aspect. The resulting algorithm, termed bilevel discretization-based feature selection (Bi-DFS), performs selection at the upper level while discretization is done at the lower level. The experimental results on several high-dimensional datasets show that Bi-DFS outperforms relevant state-of-the-art methods in terms of classification accuracy, generalization ability, and feature selection bias.

Résumé

Data imbalance is still so far a challenging issue in data classification. In literature, cost-sensitive approach has been used to deal with such a challenge. Despite its interesting results, the manual design of cost matrices is still the main shortcoming of this approach. The data engineer is still facing a great difficulty in defining the misclassification costs, especially with the absence of domain specific knowledge. Recent works suggest the use of genetic programming as an effective tool to design classification trees with automatically learned costs. Although promising results were obtained, evaluating a classification tree with a single cost matrix is not a wise choice. Indeed, the tree quality evaluation requires trying several misclassification cost matrices to be more precise and fair. Motivated by this observation, we propose in this paper a bi-level modeling of the cost-sensitive classification tree induction problem where the upper level evolves the classification trees, while the cost matrix of each tree is optimized at the lower level. Our bi-level modeling is solved using an existing co-evolutionary algorithm, and the resulting method is named Bi-COS. The obtained comparative experimental results on several imbalanced benchmark datasets show the merits of Bi-COS with respect to the state-of-the art.

Résumé

Context. This paper deals with bilateral joint decision making in supply chains, and more specifically focuses on coordinating the decisions taken by the supplier and the producer in lot sizing. Research gap. Previous existing works in lot sizing have modeled the coordination task as a bi-level optimization problem. Unfortunately, the bi-level model causes a hierarchy between the two actors by making the leader imposing the decisions that suits his/her interests to the follower. This induces a significant conflict of interest between the two stakeholders because the leaders benefit is always greater than the follower’s one. Objective. The main goal of this work is to attenuate the conflict of interest issue between both actors by proposing a multi-objective model that alleviates the hierarchy and creates a win–win situation. Method. We propose an effective multi-objective lot sizing model, called Supplier-Producer Multi-Objective Lot Sizing (SP-MOLS); that alleviates the hierarchy between the actors’ objectives by assigning them the same importance degree and hence optimizing them simultaneously. The resolution of our SP-MOLS model using the Non-dominated Sorting Genetic Algorithm II (NSGA-II), as an effective meta-heuristic search engine, provides a set of trade-off solutions, each expressing a compromise degree between the two actors: the supplier and the producer. Results. To validate our approach, we use five test problems each containing 100 instances with a planning horizon of 10 periods and we analyze the obtained trade-off solutions using the compromise degree and the gap between costs as main consensus metrics. The obtained results reveal that a small sacrifice in the leader’s benefit could produce a significant improvement in the follower’s one. For instance, a 10% increase of the producer’s cost may generate a 42% decrease in the supplier’s one. Reciprocally, a 0.4% increase of the supplier’s cost may generate a 49% decrease in the producer’s cost. Method algorithmic improvement. As solutions of interests for both stakeholders are usually located within the extreme regions of the Pareto front, we propose NSGA-II with Focus on Extreme Regions (NSGA-II-FER) as a new variant of NSGA-II that focuses the search in the extreme regions of the Pareto front thanks to a modified crowding measure that is adaptively managed during the evolution process. This variant has shown its ability to eliminate dominance-resistant solutions and thus to come up with better extreme regions. Based on the experimental results, NSGA-II-FER is shown to have the ability to provide the decision makers with more convergent and more diversified extreme non-dominated solutions, expressing better trade-off degrees between both actors’ costs. Managerial implications. The promising results obtained by our proposal encourage decision makers’ to adopt a multi-objective approach rather than a bi-level one. From our personal perspective, we recommend running the three models (the multi-objective model and the two bi-levels ones); then analyzing the solutions of all models in terms of compromise degrees and logistic costs. This would allow both actors to observe how the hierarchy incurred by the bi-level models increases conflicts, while the multi-objective one generates solutions with much improved consensus degrees. Such observations will convince the supply chain stakeholders to adopt our multi-objective approach, while keeping an eye on the bi-level models’ solutions and the consensus degrees. Finally, we also recommend focusing on the extreme regions of the Pareto front since they contain rich solutions in terms of consensus. Such solutions are more convincing in the negotiation process and thus could lead to better win–win situations.

Résumé

In many decision making cases, we may have a hierarchical situation between different optimization tasks. For instance, in production scheduling, the evaluation of the tasks assignment to a machine requires the determination of their optimal sequencing on this machine. Such situation is usually modeled as a Bi-Level Optimization Problem (BLOP). The latter consists in optimizing an upper-level (a leader) task, while having a lower-level (a follower) optimization task as a constraint. In this way, the evaluation of any upper-level solution requires finding its corresponding lower-level (near) optimal solution, which makes BLOP resolution very computationally costly. Evolutionary Algorithms (EAs) have proven their strength in solving BLOPs due to their insensitivity to the mathematical features of the objective functions such as non-linearity, non-differentiability, and high dimensionality. Moreover, EAs that are based on approximation techniques have proven their strength in solving BLOPs. Nevertheless, their application has been restricted to the continuous case as most approaches are based on approximating the lower-level optimum using classical mathematical programming and machine learning techniques. Motivated by this observation, we tackle in this paper the discrete case by proposing a Co-Evolutionary Migration-Based Algorithm, called CEMBA, that uses two populations in each level and a migration scheme; with the aim to considerably minimize the number of Function Evaluations (FEs) while ensuring good convergence towards the global optimum of the upper-level. CEMBA has been validated on a set of bi-level combinatorial production-distribution planning benchmark instances. The statistical analysis of the obtained results shows the effectiveness and efficiency of CEMBA when compared to existing state-of-the-art combinatorial bi-level EAs.

Résumé

Code smells (a.k.a. anti-patterns) are manifestations of poor design solutions that could deteriorate the software maintainability and evolution. Despite the high number of existing detection methods, the issue of class label uncertainty is usually omitted. Indeed, two human experts may have different degrees of uncertainty about the smelliness of a particular software class not only for the smell detection task but also for the smell type identification one. Thus, this uncertainty should be taken into account and then processed by detection tools. Unfortunately, these latter usually reject and/or ignore uncertain data that correspond to software classes (i.e. dataset instances) with uncertain labels. This practice could considerably degrade the detection/identification process effectiveness. Motivated by this observation and the interesting performance of the Possibilistic K-NN (PK-NN) classifier in dealing with uncertain data, we propose a new possibilistic evolutionary detection method, named ADIPOK (Anti-patterns Detection and Identification using Possibilistic Optimized K-NNs), that is able to deal with label uncertainty using some concepts stemming from the Possibility theory. ADIPOK is validated using a possibilistic base of smell examples that simulates the subjectivity of software engineers’ opinions’ uncertainty. The statistical analysis of the obtained results on a set of comparative experiments with respect to four state-of-the-art methods show the merits of our proposed method.

Résumé

Like the case of any software, Web Services (WSs) developers could introduce anti-patterns due to the lack of experience and badly-planned changes. During the last decade, search-based approaches have shown their outperformance over other approaches mainly thanks to their global search ability. Unfortunately, these approaches do not consider the uncertainty of class labels. In fact, two experts could be uncertain about the smelliness of a particular WS interface but also about the smell type. Currently, existing works reject uncertain data that correspond to WSs interfaces with doubtful labels. Motivated by this observation and the good performance of the possibilistic K-NN classifier in handling uncertain data, we propose a new evolutionary detection approach, named Web Services Anti-patterns Detection and Identification using Possibilistic Optimized K-NNs (WS-ADIPOK), which can cope with the uncertainty based on the Possibility Theory. The obtained experimental results reveal the merits of our proposal regarding four relevant state-of-the-art approaches.

Résumé

Résumé

Recently, decomposition has gained a wide interest in solving multi-objective optimization problems involving more than three objectives also known as Many-objective Optimization Problems (MaOPs). In the last few years, there have been many proposals to use decomposition to solve unconstrained problems. However, fewer is the amount of works that has been devoted to propose new decomposition-based algorithms to solve constrained many-objective problems. In this paper, we propose the ISC-Pareto dominance (Isolated Solution-based Constrained Pareto dominance) relation that has the ability to: (1) handle constrained many-objective problems characterized by different types of difficulties and (2) favor the selection of not only infeasible solutions associated to isolated sub-regions but also infeasible solutions with smaller CV (Constraint Violation) values. Our constraint handling strategy has been integrated into the framework of the Constrained Non-Dominated Sorting Genetic Algorithm-III (C-NSGA-III) to produce a new algorithm called Isolated Solution-based Constrained NSGA-III (ISC-NSGA-III). The empirical results have demonstrated that our constraint handling strategy is able to provide better and competitive results when compared against three recently proposed constrained decomposition-based many-objective evolutionary algorithms in addition to a penalty-based version of NSGA-III on the CDTLZ benchmark problems involving up to fifteen objectives. Moreover, the efficacy of ISC-NSGA-III on a real world water management problem is showcased.

Résumé

Résumé

Bi-clustering is one of the main tasks in data mining with several application domains. It consists in partitioning a data set based on both rows and columns simultaneously. One of the main difficulties in bi-clustering is the issue of finding the number of bi-clusters, which is usually a user-specified parameter. Recently, in 2017, a new multi-objective evolutionary clustering algorithm, called MOCK-II, has shown its effectiveness in data clustering while automatically determining the number of clusters. Motivated by the promising results of MOCK-II, we propose in this paper a hybrid extension of this algorithm for the case of bi-clustering. Our new algorithm, called MOBICK, uses an efficient solution encoding, an effective crossover operator, and a heuristic mutation strategy. Similarly to MOCK-II, MOBICK is able to find automatically the number of bi-clusters. The outperformance of our algorithm is shown on a set of real gene expression data sets against several existing state-of-the-art works. Moreover, to be able to compare MOBICK to MOCK-I and MOCK-II, we have designed two basic extensions of MOCK-I and MOCK-II for the case of bi-clustering that we named B-MOCK-I and B-MOCK-II. Again, the experimental results confirm the merits of our proposal.

Résumé

Decomposition-based evolutionary algorithms using predefined reference points have shown good performance in many-objective optimization. Unfortunately, almost all experimental studies have focused on problems having regular Pareto fronts (PFs). Recently, it has been shown that the performance of such algorithms is deteriorated when facing irregular PFs, such as degenerate, discontinuous, inverted, strongly convex, and/or strongly concave fronts. The main issue is that the predefined reference points may not all intersect with the PF. Therefore, many researchers have proposed to update the reference points with the aim of adapting them to the discovered Pareto shape. Unfortunately, the adaptive update does not really solve the issue for two main reasons. On the one hand, there is a considerable difficulty to set the time and the frequency of updates. On the other hand, it is not easy to define how to update the search directions for an unknown PF shape. This article proposes to approximate irregular PFs using a set of predefined normal-boundary intersection (NBI) directions. The main motivation behind this article is that when using a set of well-distributed NBI directions, all these directions intersect with the PF regardless of its shape, except for the case of discontinuous and/or degenerate fronts. To handle the latter cases, a simple interaction mechanism between the decision maker (DM) and the algorithm is used. In fact, the DM is asked if the number of NBI directions needs to be increased in some stages of the evolutionary process. If so, the resolution of the NBI directions that intersect the PF is increased to properly cover discontinuous and/or degenerate PFs. Our experimental results on benchmark problems with regular and irregular PFs, having up to fifteen objectives, show the merits of our algorithm when compared to eight of the most representative state-of-the-art algorithms.

Résumé

Multi-objective evolutionary algorithms (MOEAs) have proven their effectiveness and efficiency in solving problems with two or three objectives. However, recent studies show that MOEAs face many difficulties when tackling problems involving a larger number of objectives as their behavior becomes similar to a random walk in the search space since most individuals are nondominated with respect to each other. Motivated by the interesting results of decomposition-based approaches and preference-based ones, we propose in this paper a new decomposition-based dominance relation to deal with many-objective optimization problems and a new diversity factor based on the penalty-based boundary intersection method. Our reference point-based dominance (RP-dominance), has the ability to create a strict partial order on the set of nondominated solutions using a set of well-distributed reference points. The RP-dominance is subsequently used to substitute the Pareto dominance in nondominated sorting genetic algorithm-II (NSGA-II). The augmented MOEA, labeled as RP-dominance-based NSGA-II, has been statistically demonstrated to provide competitive and oftentimes better results when compared against four recently proposed decomposition-based MOEAs on commonly-used benchmark problems involving up to 20 objectives. In addition, the efficacy of the algorithm on a realistic water management problem is showcased.

Résumé

During the last decades, Learning Classifier Systems have known many advancements that were highlighting their potential to resolve complex problems. Despite the advantages offered by these algorithms, it is important to tackle other aspects such as the uncertainty to improve their performance. In this paper, we present a new Learning Classifier System (LCS) that deals with uncertainty in the class selection in particular imprecision. Our idea is to integrate the Belief function theory in the sUpervised Classifier System (UCS) for classification purpose. The new approach proved to be efficient to resolve several classification problems.

Résumé

During the few past years, decomposition has shown a high performance in solving Multi-objective Optimization Problems (MOPs) involving more than three objectives, called as Many-objective Optimization Problems (MaOPs). The performance of most of the existing decomposition-based algorithms has been assessed on the widely used DTLZ and WFG unconstrained test problems. However, the number of works that have been devoted to tackle the problematic of constrained many-objective optimization is relatively very small when compared to the number of works handling the unconstrained case. Recently there has been some interest to exploit infeasible isolated solutions when solving Constrained MaOPs (CMaOPs). Motivated by this observation, we firstly propose an IS-update procedure (Isolated Solution-based update procedure) that has the ability to: (1) handle CMaOPs characterized by various types of difficulties and (2) favor the selection of not only infeasible solutions associated to isolated sub-regions but also infeasible solutions with smaller Constraint Violation (CV) values. The IS-update procedure is subsequently embedded within the Multi-Objective Evolutionary Algorithm-based on Decomposition (MOEA/D). The new obtained algorithm, named ISC-MOEA/D (Isolated Solution-based Constrained MOEA/D), has been shown to provide competitive and better results when compared against three recent works on the CDTLZ benchmark problems.

Résumé

During the last decades, Learning Classifier Systems have known many advancements that were highlighting their potential to resolve complex problems. Despite the advantages offered by these algorithms, it is important to tackle other aspects such as the uncertainty to improve their performance. In this paper, we present a new Learning Classifier System (LCS) that deals with uncertainty in the class selection in particular imprecision. Our idea is to integrate the Belief function theory in the sUpervised Classifier System (UCS) for classification purpose. The new approach proved to be efficient to resolve several classification problems.

Résumé

Multi-objective evolutionary algorithms are efficient in solving problems with two or three objectives. However, recent studies have shown that they face many difficulties when tackling problems involving a larger number of objectives and their behaviors become similar to a random walk in the search space since most individuals become non-dominated with each others. Motivated by the interesting results of decomposition-based approaches and preference-based ones, we propose in this paper a new decomposition-based dominance relation called TSD-dominance (Targeted Search Directions based dominance) to deal with many-objective optimization problems. Our dominance relation has the ability to create a strict partial order on the set of Pareto-equivalent solutions using a set of well-distributed reference points, thereby producing a finer grained ranking of solutions. The TSD-dominance is subsequently used to substitute the Pareto dominance in NSGA-II. The new obtained MOEA, called TSD-NSGA-II has been statistically demonstrated to provide competitive and better results when compared with three recently proposed decomposition-based algorithms on commonly used benchmark problems involving up to twenty objectives.

Résumé

Constrained optimization is applicable to most real world engineering science problems. An efficient constraint handling method must be robust, reliable and computationally efficient. However, the performance of constraint handling mechanism deteriorates with the increase of multi-modality, non-linearity and non-convexity of the constraint functions. Most of the classical mathematics based optimization techniques fails to tackle these issues. Hence, researchers round the globe are putting hard effort to deal with multi-modality, non-linearity and non-convexity, as their presence in the real world problems are unavoidable. Initially, Evolutionary Algorithms (EAs) were developed for unconstrained optimization but engineering problems are always with certain type of constraints. The in-dependability of EAs to the structure of problem has led the researchers to re-think in applying the same to the problems incorporating the constraints. The constraint handling techniques have been successfully used to solve many single objective problems but there has been limited work in applying them to the multi-objective optimization problem. Since for most engineering science problems conflicting multi-objectives have to be satisfied simultaneously, multi-objective constraint handling should be one of the most active research area in engineering optimization. Hence, in this chapter authors have concentrated in explaining the constrained multi-objective optimization problem along with their applications.

Résumé

Multi-objective Evolutionary Algorithms (MOEAs) have proven their effectiveness and efficiency in solving complex problems with two or three objectives. However, recent studies have shown that the performance of the classical MOEAs is deteriorated when tackling problems involving a larger number of conflicting objectives. Since most individuals become non-dominated with respect to each others, the MOEAs’ behavior becomes similar to a random walk in the search space. Motivated by the fact that a wide range of real world applications involves the optimization of more than three objectives, several Many-objective Evolutionary Algorithms (MaOEAs) have been proposed in the literature. In this chapter, we highlight in the introduction the difficulties encountered by MOEAs when handling Many-objective Optimization Problems (MaOPs). Moreover, a classification of the most prominent MaOEAs is provided in an attempt to review and describe the evolution of the field. In addition, a summary of the most commonly used test problems, statistical tests, and performance indicators is presented. Finally, we outline some possible future research directions in this research area.

Résumé

Problems involving multiple conflicting objectives arise in most real world optimization problems. Evolutionary Algorithms (EAs) have gained a wide interest and success in solving problems of this nature for two main reasons: (1) EAs allow finding several members of the Pareto optimal set in a single run of the algorithm and (2) EAs are less susceptible to the shape of the Pareto front. Thus, Multi-objective EAs (MOEAs) have often been used to solve Multi-objective Problems (MOPs). This chapter aims to summarize the efforts of various researchers algorithmic processes for MOEAs in an attempt to provide a review of the use and the evolution of the field. Hence, some basic concepts and a summary of the main MOEAs are provided. We also propose a classification of the existing MOEAs in order to encourage researchers to continue shaping the field. Furthermore, we suggest a classification of the most popular performance indicators that have been used to evaluate the performance of MOEAs.