Data Preprocessing

This project is about preprocessing and preparing data for future analysis. Details are below. In this project, some steps will be done using Weka, and the others will be done using Java. The project will use a modified (red) wine dataset from UCI, although the developed project should be able to work on any dataset involving numerical attributes plus a class attribute (which is the last attribute of the file). The modification is done by mapping tuples whose quality is <=5 to class 0 and mapping other tuples to class 1. All non-class attributes are numerical. The input file is a csv file. We will call the attributes as A1, A2, …, A11, in the left-to-right order given in the input file. There are two tasks: Task 1. Discretize all attributes as follows: Use the entropy based method to split each attribute into 2 intervals; in case Weka does not produce a split for a given attribute, use the equal-density method to split the attribute into 4 intervals. The resulting bins will be stored in a csv file called Bins.csv. This file contains rows of the form for each attribute Ai with 2 bins Ai, splitValue, countC0Bin1, countC1Bin1, countC0Bin2, countC1Bin2 This file contains a row with 12 fields (3 split values + 8 counts), for each attribute with 4 bins. Task 2: Map the dataset into the itemized form using the bins produced in Task 1. This step will produce two files, namely [login to view URL] and ItemizedData.csv. The contents of these two files should follow the format given on [login to view URL] file attached. There are two possible ways to do the project: 1) Using Weka and Java, providing a Java program producing all results specified above. 2) Using Java only, providing a Java program producing all results specified above. For option 1, you should submit your Java program, and a jar file called P1.jar. The Java program should work assuming that Weka is installed on the machine. [login to view URL] when run should take the name of the dataset file as a parameter. You should assume that the input dataset is the same folder where the jar file is run. For option 2, you should submit your Java program, and a jar file called P1.jar. [login to view URL] when run should take the name of the dataset file and a minIG value as parameters. The program will use information gain (IG) to split an attribute if the IG is >= minIG, and use equal-frequency