Data-Selection

Things to keep in mind

ResNet and FAISS : convert the image dataset, into vectors and build the FAISS Index over that
- (Jiwon & Rosie)
- Dataset (CIFAR-10) : https://www.cs.toronto.edu/~kriz/cifar.html
- Download the dataset to /localdisk3/ on node 2x16a
- Go through ResNet Code : https://github.com/KaimingHe/deep-residual-networks
- Download the code and convert the CIFAR-10 images to vector embeddings
- Load the vector embeddings into a FAISS Index build on the GPU
- FAISS Index code can be found here : https://github.com/DataIntelligenceCrew/koios-semantic-search
Algorithm and Complexity Analysis : pseudocode for the algorithm and complexity analysis
- (Pranay)
- Write out the algorithm

Name		Name	Last commit message	Last commit date
Latest commit History 126 Commits
.vscode		.vscode
__pycache__		__pycache__
ds-python		ds-python
go-fkc		go-fkc
libs		libs
.gitignore		.gitignore
1024_speedup.png		1024_speedup.png
2048_speedup.png		2048_speedup.png
Makefile		Makefile
README.md		README.md
bashrc		bashrc
benchmark_data_generator.py		benchmark_data_generator.py
cifar-to-resnet.py		cifar-to-resnet.py
cifar.py		cifar.py
cifar.sh		cifar.sh
coverage_dist.png		coverage_dist.png
coverage_dist_iterative.png		coverage_dist_iterative.png
dist-alexnet.png		dist-alexnet.png
dist.png		dist.png
global_log_2.txt		global_log_2.txt
global_set_cover_alexNET.txt		global_set_cover_alexNET.txt
hw2_mpi_time.png		hw2_mpi_time.png
hw2_mpi_time_node2x14a.png		hw2_mpi_time_node2x14a.png
hw2_mpi_time_node2x18a.png		hw2_mpi_time_node2x18a.png
log.txt		log.txt
log_2.txt		log_2.txt
log_composable.txt		log_composable.txt
main		main
main.cpp		main.cpp
main.o		main.o
plot-hw.py		plot-hw.py
plot.py		plot.py
rep_quality_alexNET.png		rep_quality_alexNET.png
rep_quality_alexNET_bar.png		rep_quality_alexNET_bar.png
scatter_vectors.png		scatter_vectors.png
scatter_vectors_alexnet.png		scatter_vectors_alexnet.png
set_cover_solution_alexNet.txt		set_cover_solution_alexNet.txt
set_cover_solution_alexNet_k2.txt		set_cover_solution_alexNet_k2.txt
solution_group_distribution.png		solution_group_distribution.png
solution_group_distribution_iterative.png		solution_group_distribution_iterative.png
speedup-proj2.png		speedup-proj2.png
sqlite.py		sqlite.py
time-proj2.png		time-proj2.png