December 21, 2019

Focus your marketing efforts using clustering analysis

IntroductionIn any competitive marketplace, customer focus will be a key driver to your organisation’s success. Hence it is therefore important to understand your customer’s needs and desires rather than being fixated on the objectives of the organization e.g. increasing market share or increasing profits.Being customer focussed, we would ideally offer individual personalised service, however this can in many situations being difficult to execute and cost prohibitive. If we group our customers by their similar characteristics, we will be still be able to focus our marketing and sales efforts and in the process improve outcomes whilst being efficient with advertising funds.This activity of grouping our customers is essentially related to the activity of clustering analysis and segmentation. Performing customer segmentation enables us to answer the below questions:

Who requires the product?
Which customer requires the product?
Do these customers have any similar characteristics (age, sex, occupation, etc.,)?
What additional features / products can we create for customers who are not interested inthe current products / services on offer?
Are there any factors affecting the products being offered in the market?

ClusteringClustering (a data mining technique) is the process of grouping based on a set of defined characteristics, for example based on age, sex, demographics, etc. The clustering activity helps us to find the relationship between data points across all customers which in turn helps us create segments or groups of customers with similar characteristics.SegmentationAs mentioned above, segmentation is the activity of identifying and grouping customers based on similar characteristics. Segmenting the data will help us to identify our targeted customers for the purposes of say advertising or sales activities. For instance, let us assume that a company wants to bring a new smartphone to the market. To find out the customers we should target with advertising, it is essential to first segment the marketing data based on the customer attributes.

Are they earning more than $10,000 per month?
Do they own a house?
Do they have a good history of credit repayment?

Applying these filters will help us to identify our targeted customer groups. It is expected that customers outside this segment will not be interested in this product.Clustering and Segmentation may appear to be similar, however they are different.Segmentation: Grouping customers based on similarities.Clustering: Process of finding similarities in customers so that they can be grouped and segmented. So essentially clustering is an activity you undertake to create segments/groups.Example: Segmentation for advertising a new mobile phone in the marketScenarioA highly reputed restaurant chain is soon to be opened in the South Australian market. As a data scientist, the task is to find our targeted customers so that we can distribute advertisements via social media.BackgroundI have used the data obtained from Philippine Statistics Authority (PSA) survey based on family income and expenditure. I have used this data set as it was freely available, however this could easily be any country, but what matters here is the process.This survey, which is undertaken every three (3) years, is aimed at providing data on family income and expenditure, including, among others, levels of consumption by item of expenditure, sources of income in cash, and related information affecting income and expenditure levels and patterns in the Philippines.To keep things simple, we will be focussing only on Income and Expenditure and will be ignoring all the other variables. In real world situations, there will be many attributes or variables considered which brings added complexity and accuracy.The ApproachThe approach we have taken here, involves the following steps:

Acquire and profile customer data
Undertake (K-means) clustering
Undertake segmentation using

Step 1: Acquire and profile customer dataThe modified dataset consisting of the variables Income and Spend is being used.Income: Total household incomeSpend: Total food expenditureAll the figures included here are in Philippine peso currency. We have around 41,544 potential customer records and some statistical details is present.Review the ratio of spend to income – this provides basic understanding on how much discretionary income is available.Review the spread of data points.Visualize in three-dimensional space.Step 2: Undertake (K-means) clusteringThe primary input for k-means clustering is the number of clusters which is derived using the concept of minimizing within-cluster sum of square (WCSS). A screen plot consisting the number of clusters in X-axis and the WCSS for each cluster number in the y-axis has been shown below. As the number of clusters increases the WCSS decreases. The decrease of WCSS is initially steep, and then the rate of decline slows down resulting in an elbow plot. The findings from the Elbow method can be combined with the business requirement on how many groups are being targeted. In the graph below you can notice that there is a steep fall until cluster 4 and after that there is a steady fall. Hence, we will be considering until cluster 4.Step 3: Undertake segmentationSince we are considering a highly reputed restaurant chain, once can expect that the prices will be higher than an average meal. In the output below the data points are grouped based on income and spend. Our targeted customers are the ones in blue and orange as they spend more than average.

Summary

We have demonstrated the basic concept of clustering and segmentation by way of example. In this analysis we have considered only Income and Spend. Depending on the situation, we can include a variety of attributes e.g. age, sex, type of food, etc. Including more variables will improve the quality of the segmentation outcome, however this also increases the complexity of clustering and dimensionality reduction should be applied at that point. Other potential applicationsClustering and customer segmentation can be applied in various fields such as marketing, retail, finance, banking, image classification. What are your next steps?Do some research and consider undertaking a clustering and segmentation exercise in your company.Start small with a limited investment and then scale it out after you have proved it works.Feel free to contact Lucid Insights if you need assistance with implementing customer clustering and segmentation analytics in your organisation. To view the full source code, check on GitHub.