Hi, very interesting database challenge project !
Based on my experience this can be easily achieved.
You can have the top to resolve Your 100.000 records matching issue:
- in A.I. and image matching the data involved are even worst !
- in big data manipulation the problem is similar.
The right answer is to organize the data in the right way (e.g.: subset indexing, partial storing, direct memory mapping, logic approximation (c), there are many well known algorithms available for this )
Simply give me more details and we will overcome this challenge, reaching together the best solution needed.
Feel free to contact me for any clarification or question that can get the success of your goal
Cheers
Gianni