Verticalmove is responsible for building one of the most cutting-edge technology groups throughout Microsoft. The Experimentation and Analysis (ExP) group empowers every major product within Microsoft to objectively influence iterative changes to these products and their features through intelligent, data-driven, controlled experiments.
The internet connectivity of client software (e.g., apps running on phones and PCs), web sites, and online services provide an unprecedented opportunity to evaluate ideas quickly using controlled experiments, also called A/B tests, split tests, randomized experiments, control/treatment tests, and online field experiments. Unlike most data mining techniques for finding correlational patterns, controlled experiments allow establishing a causal relationship with high probability. Experimenters can utilize the Scientific Method to form a hypothesis of the form "If a specific change is introduced, will it improve key metrics?" and evaluate it with real users.
The unofficial shorter story...
It’s highly likely that if you use *ANY* major online service, Microsoft or not, you’re involved in an anonymous controlled experiment right now. Bing as an example conducted more than 10,000 A/B controlled experiments last year, obsessed in making Bing one of the best online experiences for it’s users in the world.
You can read further about the ExP group via these links below:
Key to a successful online business is to innovate at fast pace using controlled experiments with live users. We, the Analysis and Experimentation Team, are aiming to build a world-class experimentation service that accelerates innovations for Bing and key partners inside Microsoft by testing new ideas quickly and reliably: What factors affect the quality of the Bing user experience? What causes users to make a first payment with XBox?
The list of needs grows every day. We are hiring extremely talented, motivated and productive individuals to join us at the heart of Microsoft's innovation engine. Successful candidates will be hardcore software engineers
in at least one of the areas: API design, scalable service development, and building user interface and backend system to work with large datasets interactively. The perks with the job include the opportunities to:
• Build a software service for innovation that the world class researchers and developers in Bing, XBox and other key partner teams depend on. Apply your engineering and analytical skills on BIG DATA (petabyte scale) using one of the world's largest data processing system (COSMOS) with tens of thousands of servers. Guide the team on developing the systems for high stability, fast development, low development cost, and low maintenance cost.
• Provide technical leadership to other team members.Help to build and grow talents in these areas.
• Work directly with application teams/partners (internal clients such as Xbox, Skype, Office) to understand their offerings/domain and help them become successful with data so they can run controlled experiments (a/b testing).
• Leverage your statistical and computational knowledge to build algorithms for calculating variances.
• Understand the data generated by experiments, and producing actionable, trustworthy conclusions from them.
• Handle large amounts of data using various tools, including your own. We prefer C#, Python and SQL but are open to all OO programming languages.
• Build data manipulation, data processing, and data visualization tools and share these tools across Microsoft.
• Apply data analysis, data mining and data processing to present data clearly and develop experiments (ab testing).
• Ensure high-quality data and understand how data is generated out experimental design and how these experiments can produce actionable, trustworthy conclusions.
• Work with development team to build tools for data logging and repeatable data tasks to accelerate and automate data scientist duties.
• Assist senior management in making key business decisions.
• Experience in statistical data analysis at web-scale.
• Experience in user interface design.
• Familiarity with data analysis tools and statistical packages, such as R and Matlab.
• Strong design and problem-solving skills, with a strong bias for engineering excellence at scale with a track record of shipping multiple releases.
• Expert coding skills in C++, Java, C# (5+ years).
• Extensive knowledge of database design and programming.
• Excellent written and verbal communication skills.
• Bachelor's or Master's degree in Computer Science or a related field.