Verticalmove is responsible for building one of the most cutting-edge technology groups throughout Microsoft. The Experimentation and Analysis (ExP) group empowers every major product within Microsoft to objectively influence iterative changes to these products and their features through intelligent, data-driven, controlled experiments.
The internet connectivity of client software (e.g., apps running on phones and PCs), web sites, and online services provide an unprecedented opportunity to evaluate ideas quickly using controlled experiments, also called A/B tests, split tests, randomized experiments, control/treatment tests, and online field experiments. Unlike most data mining techniques for finding correlational patterns, controlled experiments allow establishing a causal relationship with high probability. Experimenters can utilize the Scientific Method to form a hypothesis of the form "If a specific change is introduced, will it improve key metrics?" and evaluate it with real users.
The unofficial shorter story...
It’s highly likely that if you use *ANY* major online service, Microsoft or not, you’re involved in an anonymous controlled experiment right now. Bing as an example conducted more than 10,000 A/B controlled experiments last year, obsessed in making Bing one of the best online experiences for it’s users in the world.
You can read further about the ExP group via these links below:
The platform is to initially enable experimentation everywhere our client has an online presence, whether website or services, then to take experimentation to the developer ecosystem. As a Program Manager, you will be responsible for helping to shape the strategy, define the platform, and interact with our customers (both internal and later the development community). You will write specifications to be used by developers and testers, and help manage projects to completion in a timely manner with high quality.
We look for candidates that have a solid technology background, entrepreneurial spirit, and good cross-team interaction skills.
- Partnering and onboarding new teams to the ExP platform
- Developing new experimentation service capabilities and
- Analyzing experimentation service use and optimizing the
- Designing tools to scale analysis activities
- Optimizing experimentation quality and processes
Extra Consideration give to those with experience in:
-Background in highly scalable systems
-Background in Statistics and/or Machine Learning/Data-Mining
-Database experience (SQL/OLAP)
- Strong technical background and strategic thinking
- Passion for data driven decision making
- Experience designing and operating online services including defining and improving operational metrics
- Experience in online user behavior research including instrumentation, metrics, log analysis and experiment design
- Demonstrated enterprise-level program management skills
- Familiarity with statistics - SQL, Excel and reporting systems experience
- Excellent communication and cross group collaboration skills
- 10+ years of Program Management experience delivering online services within a collaborative environment
• Bachelor’s or Master’s degree in Computer Science, Math, Physics, Engineering, Statistics.