DS 121: Foundations of Data Science II

The goal of the final project is to give you the opportunity to further explore one of the topics covered in the course. Concretely, your objective is to analyze a dataset using any of the techniques we have covered this semester such as clustering, regression, principal components analysis, matrix factorization, or more.

You should find a dataset from a research paper, a Python repository, a blog post, or any other source (see some suggestions below). Then, you must either run a new analysis from scratch on this dataset, or you can reproduce the result of a prior work and add at least one extension. Note that it’s perfectly fine to use ideas from other papers or websites, but you must (a) cite any sources used and (b) describe concretely the parts of your project report that are similar to, or different than, the prior work.

Project Report Description: Your final report should be a Jupyter notebook containing the following sections:

1. Introduction: Provide an easy-to-understand summary of what you’ve done and why it matters to you. This section should be understandable even by people who have not taken this course.
   
   

2. Data: Describe the dataset you have studied. Explain what the objects and features are, show some samples, etc. If appropriate, describe where the dataset comes from and where it is applied in practice.
   
   

3. Methodology: Here, you should state the analysis techniques that you used in the project. Make sure to explain what the algorithm you’re using does, and why you chose this particular strategy for analyzing the data. If appropriate, state a hypothesis that you plan to test.

4. Analysis: Show the analysis itself, and include any charts/graphs/tables that help to visualize what you’ve done. This section should contain the code for implementing your methodology. For example, a project using matrix factorizations could include an implementation of one or more of the factorizations we discussed in class (or a related one).
   
   

5. Results: Explain the takeaways from your analysis. For example, does your analysis support or refute the claim you were intending to study, and did your algorithm behave as expected? If your project is building upon someone else’s work, make sure to use this space to compare and contrast your findings with other works.
   
   

6. Conclusion: Summarize the work you’ve done and the outcomes you’ve discovered.
   
   

7. References: I’ll repeat, make sure to cite your work! You can use any textbook, website, paper, or other resource as long as you cite it. Using prior work without citing it is plagiarism and will be handled as stated in the course syllabus.
   
   

Here is the timeline for the project:

• Wednesday, November 30: By this day, please send me a private Piazza note describing the project that you plan to pursue. That is, describe the dataset you want to analyze and your initial ideas for the kinds of data analysis you think would be relevant to understanding this dataset.

• Monday, December 12: Submit your final project on Gradescope.

The proposal: This is a brief summary of what topic you would like to tackle for the project. Deciding and scoping what exactly you want to do is very much part of the project itself. I am happy to chat during office hours or by appointment about ideas you may have and provide relevant references. The proposal should provide an overview of the topic you are choosing.

Your project will be graded as follows:

• 5% for submitting the initial project plan by Nov 30. (You will get full credit here for submitting any plan by Nov 30, even if we have some discussion about it afterward.)

• 15% for each of the six sections in the report. We will grade each section based on how well your code uses the techniques learned in class, and how well your writing documents this work.

• 5% for including a references section. (See more details below.)

Remember that it is crucial to include a references section and to properly compare your project with any prior work. Your bibliography entries should include author(s), title, and a website link (if appropriate).

Copying of ideas from prior work without citation will be considered plagiarism, and it is grounds for receiving a grade of 0 on the project along with further disciplinary action as described in the syllabus. As a general rule: cite liberally whenever you quote or paraphrase ideas from another paper, in every single sentence/paragraph where you do so. If you have any questions about this policy, send a private Piazza note to ask us.

Resources: 

Algorithms and Libraries:- Useful libraries for data science in Python include:

[Machine Learning]

• Scikit-learn

• TensorFlow

• Keras

[Data Processing]

• Pandas

• NumPy

• SciPy

[Visualizations]

• matplotlib

• Seaborn

• NetworkX

Examples of applications can be found on:

• Analytics Vidhya

• Machine Learning Mastery

Datasets:- You can use any open dataset. Some common sources include:

[Dataset aggregators for Machine Learning projects]

• Kaggle

• Data Flair

• OpenML

• UCI Machine Learning Repository

[Open Data]

• data.europa.eu

• World Bank Open Data

• DATA.GOV

• HealthData.gov

This page contains the lesson plan for DS 121 lectures.