We have a list of domains about 100,000,000 in total, I imagine about 50% are duplicates. We need to process the entire list and remove the duplicates.
The final output should be a list of UNIQUE domains..
I have been processing using EMeditor for the past 48 hours on a an i7 PC with 16GB of RAM, and it's still no where near finished.
We need some massive power to process this.
Please do not bid unless you have worked with data of this size before.
44 freelancers are bidding on average $147 for this job
I can remove duplicates from 100 million lines of domains list and output the unique lines in sql or text format. I will complete this work in 2 days. Looking for your reply to start this work immediately.
Hello sir,I have 8 member team and good experience and we can start the work right now and also all communication and work will be high quality. all work will be on my office without any [url removed, login to view]