Capstone – Project

Generating And Testing Hypotheses About User Retention Using Site Analytics

A Collaboration Between

Project Synopsis

The objective of this capstone project is to generate hypotheses about which demographic and behavioral segments are correlated with user retention, to test these hypotheses using the tools available within Google Analytics, and to produce actionable reports useful to Marketing and/or Product departments aiming to increase user retention and ultimately grow the user base. Students will gain facility with Google Analytics, will exercise their data analysis and visualization skills, and will find ways to effectively, succinctly, and creatively communicate results to other departments within a firm. The first milestone entails gaining familiarity with a community forecasting platform before producing a table of variables hypothesized to be positively or negatively correlated with user retention across a 12-week period in the life of the platform, along with brief explanations of why these relationships might exist. The second milestone requires a dive into the different types of user and session-level segmentation made possible by Google Analytics, followed by creation of a more refined table of variables to be tested, as well as plausible means for creating segments able to test these variables. The third milestone calls for creating these segments within Google Analytics and analyzing the resulting retention data while keeping careful record of which variables did or did not correlate with retention, and while using these results to fine-tune the first set of segments to run a second set of analyses. The fourth milestone is a final report to include visualizations of data pulled directly from Google Analytics, as well as plots of data generated from the segmentation analyses. This report will home in on the most salient, surprising, and actionable results from the segmentation analyses, will consider why these variables were or were not correlated with retention, and will offer suggestions to both Marketing and Product departments on ways they might make use of the results from the analyses.

Project Topics

Marketing

Product Design & Development

Reporting, Financial Planning & Analysis

Company Information

CompanyMetaculus
HQSanta Cruz, CA
RevenueUnlisted
EmployeesUnlisted
StageHigh-Growth Startup
Hiring PotentialN/A
Website

Company Overview

Metaculus is a community forecasting platform that delivers ML-optimized collective intelligence on topics of global importance. The platform brings together a global reasoning community, and keeps score for these thousands of forecasters via explicit, standardized track records. Built on data, it generates scalable ensemble forecasts while delivering consistent feedback on the quality of the reasoning and analysis of its participants. The collective intelligence generated by Metaculus connects rigorous modeling and the scientific method to complex real-world challenges, thereby improving decision-making and fostering transparency and accountability in the public domain. Its open access ensemble forecasts provide a sense-making utility for hundreds of thousands of readers, and are trusted by the scientific community, government agencies, effective charities, and the Fortune 500.

Experiential Learning Program Details

SchoolUniversity of South Carolina – Upstate
Engagement Format -
CourseSummer Capstone in Business Analytics
Level
  • All Graduate
Students Enrolled40
Meeting Day & TimeOnline
Student Time Commitment4-7 Hours Per Week
Company Time Commitment1 Hour
Duration8.29 Weeks

Program Timeline

Touchpoints & Assignments Due Date Type
REMINDER OF CONFIDENTIALITY

REMINDER OF CONFIDENTIALITY

During this program, you'll be working on a real project posed by a real industry partner. By registering for the CapSource system and participating in this Program, you are bound by the Rules of Engagement, including strict confidentiality of information shared with you. Please be respectful and professional.
June 4th, 2021 Event na
Complete Pre-Kickoff Evaluation

Complete Pre-Kickoff Evaluation

June 6th, 2021 Event na
OFFICIAL PROJECT KICKOFF

OFFICIAL PROJECT KICKOFF

Find time to web conference and discuss project goals and timeline
June 14th, 2021 Event na
Complete Temp Check Evaluation

Complete Temp Check Evaluation

June 28th, 2021 Event na
Complete Temp Check Evaluation

Complete Temp Check Evaluation

July 14th, 2021 Event na
OFFICIAL PROJECT END

OFFICIAL PROJECT END

August 4th, 2021 Event na
Complete Final Peer Evaluation Survey

Complete Final Peer Evaluation Survey

August 7th, 2021 Event na
Complete Final Self Evaluation Survey

Complete Final Self Evaluation Survey

August 7th, 2021 Event na

Key Project Milestones

  • June 28, 2021 - Product Analysis and Hypothesis Building

    The first milestone entails gaining familiarity with a community forecasting platform before producing a table of variables hypothesized to be positively or negatively correlated with user retention across a 12-week period in the life of the platform, along with brief explanations of why these relationships might exist.


    Suggested Deliverable:

    A Word or Google doc detailing at least ten hypotheses of relationships that might exist between a particular variable and user-retention, as well as whether that relationship is positive or negative. There should also be a link to a relevant page of the site and an explanation of what on that page prompts a given hypothesis. If there are other offsite links that provide more general rationales for looking at this or that variable in light of user retention, these can be provided.

  • July 12, 2021 - Skill Up in Google Analytics Toolkit and Refine Hypotheses

    The second milestone requires a dive into the different types of user and session-level segmentation made possible by Google Analytics, followed by creation of a more refined table of variables to be tested, as well as plausible means for creating segments able to test these variables.


    Suggested Deliverable:

    A Word or Google Doc updating on the first milestone deliverable to explain for each of the first document’s proposed hypotheses why it is or is not practical to be able to test a given hypothesis. For those that can be tested, a description should be provided of several segments that the student will generate in order to analyze the resulting cohort data. For any hypotheses that are no longer viable, the student should generate a new hypothesis to take its place until the student has at least 10 different hypotheses to test, each with at least one proposed segment to divide the data on.

  • July 23, 2021 - User and Session-Level Segment Creation, Logging, and Additional Segmentation

    The third milestone calls for creating these segments within Google Analytics and analyzing the resulting retention data while keeping careful record of which variables did or did not correlate with retention, and while using these results to fine-tune the first set of segments to run a second set of analyses.


    Suggested Deliverable:

    An Excel workbook or Google spreadsheet with a row for each segment created. One column indicates the hypothesis to be tested. Another column indicates the segment used. Another column indicates the result of the analysis.

    A Word or Google doc writing up the results of the analyses and proposing additional segments to test the hypotheses as needed, in light of information the student gained in the first set of analyses. These additional segments are added as rows in the Excel workbook or Google spreadsheet as a separate tab or sheet.

  • August 4, 2021 - Final Report with High-Level Writeup of Results of Analyses, Data Visualizations, and Actionable Next Steps

    The fourth milestone is a final report to include visualizations of data pulled directly from Google Analytics, as well as plots of data generated from the segmentation analyses. This report will home in on the most salient, surprising, and actionable results from the segmentation analyses, will consider why these variables were or were not correlated with retention, and will offer suggestions to both Marketing and Product departments on ways they might make use of the results from the analyses.


    Suggested Deliverable:

    A Word or Google doc at least 1000 words in length and with at least two plots or data visualizations. This report will detail the results of the analyses conducted in a high-level manner to focus on the most salient, surprising, and useful results. The report will also provide actionable next steps for Marketing and/or Product departments in light of the results of the analyses. That is, what change to the product would likely increase retention and why? What might an effective marketing campaign look like and why?

Project Resources

There are no resources currently available

Industry Mentors

Company Admin

Christian Williams

[email protected]

Academic Mentors

There are currently no supervisors assigned.

Assigned Students

There are currently no students assigned.