Photo by Emile Perron on Unsplash

Combining multiple recommendation system techniques to effectively predict the users’ habit

Recommendation System

In simple words, Recommendation system is to predict the user’s habits.

As the technology advances, the customer behaviors have changed, so does the way people search for their desired products, online shopping has dramatically changed the retail ecosystem. The number of products is increasing fast, but only small portion of the product gets the attention, which cause the long-tail effect. Recommendation System can help to advertise the item with lower popularity.

In practice, it is hard to find a single model that is tailored to fit our dataset. Most…


Photo by Ian Schneider on Unsplash

Item-based model is widely implemented in recommendation system, however, it is not good enough to merely implement the item-based model. Usually, we need to explore more on the model and combine with some feature engineering technologies and user behavior analysis. In this example, we will continue from the previous post, and explore more on the item-based model extensions.

Github Link: https://github.com/chiang9/Recommendation_system_pyspark/blob/main/Item_based_cf/Extensions_of_Item-based_model.ipynb

Please check out the previous post,

Recap

We will use the following formula to make prediction


Collaborative Filtering is widely used in building recommendation system. There are 2 main approaches in memory-based model, item-based and user-based. In practice, we usually have data with more users than the number of items and only a small portion of items get frequent rating, and this situation is called the long tail effect. The long tail effect usually implies the high data sparsity, and sparse data will affect the accuracy of our model.

In order to resolve the high sparsity issue, Item-based model often leads to faster online learning and better recommendation. …


Matrix Factorization is widely used in recommendation system, we are going to build with the als model, and explain the math behind it

Collaborative Filtering is the most implemented and mature recommendation system. We are going to build the recommendation system with model based — matrix factorization, using the ALS model provided by pyspark.

GitHub Link: https://github.com/chiang9/Recommendation_system_pyspark/blob/main/ALS_model/movielen%20ALS.ipynb

Figure 1. Utility Matrix

A sparse matrix R can be built based on the data of user-item relation and their ratings. The scores can be obtained from the users or the user behaviors. Collaborative Filtering method uses the matrix R to produce the recommendations.

Jeffery chiang

Data Engineer working on recommendation systems

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store