GoodPro App ( New )

Train Folder

I am making three datasets: train1, train2, and train3. Train1 has already been cleaned to remove unnecessary columns, train2 is left untouched since I didn’t do time series analysis (it would take me too long to explain to you why), and the third one is the testing dataset and will be split into two: testing (we’re doing feature engineering here) and unseen (we’ll talk about the latter later on). The goal of having these different kinds of datasets is that you need to have two datasets to feed into a pipeline and have the same rows on both data sets, otherwise the model won’t scale right and you end up with mismatched datasets. My original plan was to add some features to the training dataset, so I could load it again (to fit a larger sample of train2 ) or even create multiple instances of data (to fit a smaller subset of train2 ) and add those rows to the trained1 and tested3 datasets. A quick google search showed me how hard it is to do such analyses without actually going through the code, so I’ll skip that. Fortunately some libraries (like statsmodels) let us combine data frames and that is basically what I used to accomplish this. If I had known there wouldn’t be any preprocessing (like removing missing values, or grouping data by categories or such) that would allow to do this analysis I wouldn’t have bothered.

If I hadn’t done all of this during the second half of 2020 I wouldn’t have been able to get it completed at all if I hadn’t also done it during the spring break semester I spent at UW. At least I would not have had all of the dependencies installed on the local machine I’m renting until late June and would have had some issues installing. Those were tough lessons I learned, especially when you learn something that is new and it is not compatible with something you did before.

A screenshot from one of my notebooks

That will get you started if you have access to a Unix terminal and know how to type things up there. However, if you are new to Linux or MacOS, a Mac or PC, or another operating system, you do not need to worry about connecting directly to an OS. You can install everything in Git, clone the repository, fork the repo, move to a new directory, clone it, and start working on a new notebook.

First few lines from an initial version of a notebook (they might change)

Here are the instructions for getting it ready for publishing: https://github.com/pierrondott/py_sphark_notebook

It gets more straightforward from there. Yes, this first step is not as tricky as it seems, but some times what you can achieve with these tools is way outside your grasp. As a beginner it feels like you can write whatever you want, but as your knowledge grows you might be discouraged and think that it isn’t worth it to figure it all yourself. That is all just sad truth, especially when you can rely on help from StackOverflow or Quora to pull you through.

After you have uploaded a notebook please leave it alone. Someone else can check and review it and see if it’s good or not. Keep the link on the gist, but don’t edit it, make it public, or anything else you think might confuse anyone who may see it in their future when you upload new stuff. When I see something in a notebook I try to make a comment about it on the web, but only after someone notices the link (I don’t really like it, but at least I’m keeping a copy of the whole thing for reference). I tried to make a note to include the exact same thing I did before (there is no documentation on GitHub for those), but I forgot about it and didn’t see it.

There are several ways to reach out if you need help building a similar project as mine. One option is obviously Medium or Twitter. I usually go on a random walk to my spot and blog about it on a daily basis. I’d love to hear of someone who uses this approach too.

If someone likes what they see here let them share it. I’m making some changes with this version after writing this post .

( HOW TO DOWNLOAD APPLIACTION )

Open File Explorer and find the zipped folder.To unzip the entire folder, right-click to select Extract All, and then follow the instructions.To unzip a single file or App, double-click the zipped folder to open it. Then, drag or copy the item from the zipped.