Modernization Hub

Modernization and Improvement
Power BI & Azure Data Services – Better together – BRK3086

Power BI & Azure Data Services – Better together – BRK3086


CHRISTIAN WADE. WE HAVE JUSTYNA LUCZNIK WITH US. WE ARE PROGRAM MANAGERS FOR BI. WE ARE GOING TO TALK ABOUT POWER BI AND AZURE DATA SERVICES WORKING TOGETHER TO PROVIDE AN END TO END SOLUTION THAT WORKS FOR DIFFERENT PERSONAS PERSONAS IN AN ENTERPRISE ORGANIZATION. SO RATHER THAN, WE ARE GOING TO BE DEMO INTENSIVE. BUT RATHER THAN, THIS IS THE DEMO FOR THE DATES ENGINEERED. THIS IS THE AZURE ML DATA SCIENTIST. AND THE POWER BI DATA FLOW DEMO FOR THE BUSINESS END, WE ARE GOING TO DO THE DEMOS AS DIFFERENT PERSONAS USING END TO END SYSTEM AND SHOW HOW THE SYSTEM WORKS TOGETHER ACROSS AN AZURE OVERALL SOLUTION, ACROSS DIFFERENT PERSONAS IN ENTERPRISE ORGANIZATION. OKAY. ALL RIGHT. SO IN ORDER FOR AN ORGANIZATION TO REALLY LEVERAGE POWER BI AND AZURE DATA SERVICES AS END TO END PLATFORM TO REALLY MAKE THE MOST OF IT, THESE ORGANIZATIONS NEED A DATA CULTURE SO I WOULD LIKE TO JUST COVER SOME OF THE ATTRIBUTES OF ORGANIZATIONS SEEKING TO EMBRACE A DATA CULTURE SUCCESSFULLY. AND FIRST OFF, DATA LITERACY IS OBVIOUSLY A BIG DEAL. YOU CAN CHIME IN. YOU CAN KEEP IT FAIRLY INTERACTIVE. DATA LITERACY AND THE WORKFORCE HAVING THE NECESSARY TOOLS TO ACCESS A WIDE VARIETY OF DATA SOURCES, MANIPULATE DATA, PREPARE DATA, VISUALIZE DATA, WE LIKE TO THINK OF POWER BI AS THE POWERPOINT FOR DATA THAT REALLY ENABLES EMPLOYEES TO PRESENT THEIR CASE BASED ON FACTS RATHER THAN KIND OF OPINION BASED DECISION-MAKING. ANOTHER ATTRIBUTE OF SUCH AN ORGANIZATION WOULD BE EASE OF DATA SHARING SO THERE ARE A FEW DIFFERENT FLAVORS OF DATA SHARING. WE HAVE, FOR EXAMPLE, PAIR TO PAIR SHARING WHERE BUSINESS ANALYSTS AND PEOPLE IN THE BUSINESS CAN COLLABORATE AND SHARE DATA ARTIFACTS EASILY USING A SYSTEM THAT ALLOWS THEM TO SHARE THAT ARTIFACT EASILY AND BUILD ON EACH OTHER’S WORK AND NOT REINVENT THE WHEEL TO SOME DEGREE AND WE ALSO HAVE THE TRADITIONAL KIND OF DATA WAREHOUSING AND CORPORATE BI DATA SHARING WHERE IT WILL CREATE THESE CURATED DATA ARTIFACTS THAT GO THROUGH MORE EXTENSIVE LEVEL OF RIGOR IN TERMS OF DATA CLASSIFICATION AND DATA GOVERNANCE AND INFORMATION ARCHITECTURE AND GETTING DEPLOYED ACROSS ENVIRONMENTS AND THEY HAVE APPLICATION LIFE-CYCLE MANAGEMENT REQUIREMENTS FOR MISSION CRITICAL SYSTEMS. THOSE, THEY KIND OF DRIVE CORPORATE METRICS THAT HAVE REUSED THROUGHOUT ENTERPRISE ORGANIZATIONS. SO ALL OF THE ABOVE FITS INTO AN ORGANIZATION THAT REALLY EMBRACES DATA CULTURE AND REALLY IT IS MORE OF A CULTURAL THING THAN A TECHNOLOGY THING. THE TECHNOLOGY FACILITATES THE PRODUCTIVE USE OF DATA IN AN ORGANIZATION. I THINK THAT WE, AT MICROSOFT HAVE A PLATFORM THAT IS VERY WELL DIFFERENTIATED IN THAT RESPECT IN TERMS OF FACILITATING ORGANIZATIONS TO REALLY GET UP TO SPEED ON A DITA CULTURE. OKAY. ANY QUESTIONS? COMMENTS? NOPE. OKAY. SO THIS IS LET’S SEE, THIS IS BASICALLY A SUMMARY OF WHAT I WENT THROUGH. AND WE ALSO WILL COVER WHAT ARE SOME OF THE CHALLENGES THAT FACE ORGANIZATIONS SEEKING TO EMBRACE THE DATA CULTURE. FIRST OFF, IS DATA SILO, RIGHT? DATA FRAGMENTATION THAT IS POISONOUS TO A DATA CULTURE AND YOU, YOU SEE THIS QUITE A LOT WITH THESE BIG ENTERPRISE CUSTOMERS BECAUSE THERE ARE SO MANY DIFFERENT SOFTWARE VENDORS FOR SO MANY DIFFERENT NICHES. YOU HAVE SOFTWARE VENDORS THAT SPECIALIZE IN BIG DATA AND SMALL DATA AND ENTERPRISE BI AND SELF-SERVICE BI AND AI AND ETC. YOU GET THE DIFFERENT ORGANIZATION, THE DIFFERENT DEPARTMENTS IN THE ORGANIZATION. THEY ALL HAVE THEIR OWN LITTLE PREFERENCES. LARGE CUSTOMERS END UP WITH A PLETHORA OF DIFFERENT SOFTWARE PACKAGES AND EACH OF THEM END UP WITH THEIR OWN LITTLE DATA SILOS AND THEIR OWN LITTLE POCKETS OF DATA THAT DON’T INTEGRATE VERY WELL AND OFTEN STORED IN DIFFERENT FORMATS IN PRIORITY FORMATS AND DIFFERENT SECURITY MODELS AND DIFFERENT CLOUDS AND IF YOU TRY TO INTEGRATE ALL OF THAT TO CREATE HOLISTIC HOLD FOR YOUR ORGANIZATION IT IS A REAL CHALLENGE. YOU HAVE INGRESS CHARGES ACROSS DIFFERENT CLOUDS AND SECURITY MODEL. IT JUST BECOMES A MESS. THIS IS WHERE I THINK THAT MICROSOFT IS UNIQUELY POSITIONED TO PROVIDE AN END TO END SOLUTION WHERE YOU HAVE A UNIFIED ARCHITECTURE OR SINGLE SECURITY MODEL, THE DATA. THERE SHOULDN’T BE ANY, IN TERMS OF ACCESS AND DATA, THERE SHOULDN’T BE ARCHITECTURAL BARRIER, RIGHTED? YOU SHOULDN’T NEED TO COPY DATA ACROSS CLOUDS. EVERYTHING RIGHT THERE. THE ONLY BARRIER SHOULD BE SECURITY AND PRIVILEGE ASY CONTROLS AND THAT IS IT. WE ARE TARGETING THIS UNIFIED ARCHITECTURE WITH AZURE DATA AND POWER AI THAT IS WHAT WE CAN PROVIDE. OKAY. ANY COMMENTS? QUESTIONS? ANYTHING? ALL RIGHT. SO HOW DOES POWER BI BUILD ON THIS UNIFIED PLATFORM? SO FIRST OFF, POWER BI WORKS FOR BOTH SELF-SERVICE AND ENTERPRISE BI. SO I’M NOT SURE HOW FAMILIAR YOU ARE WITH THESE TERMS BUT YOU KNOW, ENTERPRISE BI IS THE TRADITIONAL FORM OF BUSINESS INTELLIGENCE WHERE IT CREATES THESE CURATED DATA ARTIFACTS FOR WIDE REUSABILITY AND WE AT MICROSOFT HAVE A DEEP HERITAGE IN ENTERPRISE BI AND ONE OF THE KEY PRODUCTS IN THE SPACE IS CALLED ANALYSIS SERVICES, REPORTING SERVICES THAT YOU MAY BE FAMILIAR WITH. IT IS 20 YEARS OLD. 20 YEAR HERITAGE IN POWER BI. ONLY VENDOR TO BE RANKED IN THE LEADERS QUADRANT IN MAGIC QUADRANT FOR 12 YEARS. 12 YEARS AGO, SELF-SERVICE BI DIDN’T EXIST AS A DISCIPLINE. NONE OF THESE MODERN BI WERE ON THE RADAR AT THAT POINT. WE HAVE HERITAGE LIKE NO OTHER VENDOR. AND WE WERE THE ONLY TRADITIONAL BI VENDOR TO REALLY EMBRACE SELF-SERVICE BI EARLY ON WITH POWER BI. AND WE NOW HAVE A UNIFIED PLATFORM THAT PROVIDES, YOU KNOW, THE BEST OF BOTH WORLDS ON A SINGLE ONE STOP SHOP THAT ALLOWS THE BUSINESS IT TO COLLABORATE. THEND USER, MOST END USERS WANT TO SEE THE DATA IN THE REPORT. THEY DON’T KNOW WHETHER THIS WAS PREPARED BY SOMEONE IN IT, OR THEY JUST WANT TO SEE THE DATA AND SINGLE ENTRY POINT FOR WORKING FOR BOTH THE TYPE OF BI SCENARIOS IS A REAL, JUST SIMPLIFIES THINGS A LOT. OKAY. THEN WE HAVE ALSO GOT, WE ALSO WORK IN TERMS OF DATA PREP FOR THE UNIFIED DATA. SO ALL OF THE DEMOS THAT JUSTYNA AND I ARE GOING TO SHOW, THEY ALL READ AND WRITE TO A SINGLE UNIFIED DATA LEG BASED ON ALSGEN2. THAT IS SINGLE UNIFIED ARCHITECTURE WHERE YOU DON’T NEED TO COPY DATA ACROSS CLOUDS. EVERYTHING RIGHT THERE. ONLY LIMITED BY SECURITY AND PRIVACY. NOT ONLY THAT, ALL OF THE TOOLS AN TECHNOLOGIES AN PRODUCTS THAT WE ARE GOING TO DEMO, CW, AZURE ML, DATABRICKS, ADF, ALL OF THESE TOOLS AND PRODUCTS THEY ALL READ AND WRITE DATA IN A STANDARD FORMAT CALLED THE COMMON DATA MODEL WHICH IS SOMETHING THAT SATYA MENTIONED IN HIS KEYNOTE YESTERDAY. THE WHOLE POINT THERE IS THAT ALL OF THESE TOOLS AND PRODUCTS CAN READ AND WRITE IN THE SAME OPEN STANDARD FORMAT. NO PROPRIETY FORMAT. THIRD PARTY, NON-MICROSOFT TOOLS COULD COME AND READ AND WRITE THAT DATA IF THEY WANT TO. TO MAKE THAT DATA INTO CHANGE ACROSS THE DIFFERENT PRODUCTS AND THEREFORE ACROSS THE DIFFERENT PERSONAS THAT MUCH MORE STRAIGHTFORWARD. OKAY . LAST BUT NO MEANS LEAST, PERVASIVE AND UNIFIED AI FOR BI. SO SOME OF THE DEMOS THAT JUSTYNA ARE GOING TO SHOW ARE ABSOLUTELY MIND BLOWING, YOU KNOW, IT IS REALLY EXCITING TO SEE THAT WE ARE REALLY AUGMENTING BUSINESS INTELLIGENCE WITH AI IN REALLY PHENOMENAL WAYS. AND THE WHOLE, WHOLE THING HERE IS JUST MAKING AI ACCESSIBLE TO NON-TECHNICAL BUSINESS USERS. PUTTING IT RIGHT THERE IN THE TOOLS AND PRODUCTS THAT THEY USE. MAKING IT APPROACHABLE, RIGHT? AS JUSTYNA IS GOING TO SHOW, PUTTING RIGHT THERE IN POWER BI FOR THE BUSINESS ANALYST AND MAKING IT EASY TO CONSUME, THAT IS THEREFORE GOING TO INCREASE THE DEMAND FOR AI. SO WE THEREFORE NEED TO INCREASE THE SUPPLY FOR AI AS WELL. AND YOU KNOW, AND BECAUSE TRADITIONALLY DATA SCIENCE MIGHT HAVE BEEN KIND OF RESTRICTED A LITTLE BIT TO THE DATA SCIENTIST COULD BECOME A BIT OF A BOTTLENECK IN THE ORGANIZATION BECAUSE IT IS QUITE TEDIOUS REPETITIVE WORK TO TRAIN THESE AI MODELS. TO ACCELERATE THE DELIVERY OF AI, WE USE AUTO ML TO AUTOMATICALLY TRAIN THESE MODELS AND IDENTIFY THE OPTIMUM MODEL. WITH AUTO, THERE IS MORE MODEL TRAINING GOING ON, YOU WANT TO USE THE CONFRONTATIONAL POWER WITH AZURE TO DELIVERY OF AI TO MEET THE DEMAND. OKAY. MAKE SENSE? COMMENTS? QUESTIONS? FEEDBACK? GO AHEAD .>>[INAUDIBLE]>>SO THE QUESTION, THANK YOU FOR THE QUESTION. THE QUESTION IS, ANYMORE INFORMATION ABOUT THE COMMON DATA MODEL THIS SESSION? YES. JUSTYNA WILL BE GOING INTO THAT IN QUITE A BIT OF DETAIL AND I’LL BE COVERING IT A LITTLE BIT AS WELL. AND WE CAN REFER YOU TO THE REALLY DEEP DIVE, RESOURCE, MAYBE IF YOU COME AND TALK TO US AFTER THE SESSION, WE CAN PUT YOU IN CONTACT WITH THOSE AS WELL. YES WE WILL BE DRILLING INTO THE COMMON DATA MODEL. OKAY. ANY OTHERS? NO. YES, GO AHEAD.>> [INAUDIBLE]>>YES, YES.>>I [INAUDIBLE] >>SO THE QUESTION WAS AROUND THE USING OF THE COMMON DATA MODEL FORFORMAT IN THE DATA WITH THE NATIVE ACCESS PROTOCOL FORCE THE DATA LINK. SO YES WE WILL BE SHOWING THAT. SO THE COMMON DATA MODEL BASICALLY JUST STORES THE DATA IN THE DATA LINK WITH CFS MODEL THAT DESCRIBES THE SCHEMER SO THESE TOOLS WILL HAVE NATIVE CONNECTORS THAT READ AND WRITE IN THAT FORMAT THAT MAKES IT INTERCHANGEABLE ACROSS THESE APPLICATIONS. OKAY. ALL RIGHT. AND JUST BEFORE WE JUMP INTO THE DEMOS, IT IS WORTH POINTING OUT THAT, YOU KNOW, THIS IS THE NEW QUADRANT AS OF FEBRUARY, AS YOU CAN SEE, MICROSOFT IS OVER THERE IN THE TOP RIGHT. THIS IS FOR BI AND ANALYTIC PLATFORMS. WHERE THE ONLY VENDOR AS I SAID WHO, TO BE RANKED IN THE LEADERS QUADRANT FOR THE LAST 12 YEARS AND THE REASON THAT WE ARE RECOGNIZED BY RESEARCH AGENCIES BY GARDNER IS PRECISELY BECAUSE WE EMBRACE THESE WAVES EARLY ON. RIGHT. WE ARE DRIVERS OF THE WAVES. THINGS LIKE AUGMENTING BI WITH AI. THINGS LIKE THE CONVERGENCE OF SELF-SERVICE AND BI AND SINGLE ALL ENCLUESIVE PLATFORM. OKAY. ALL RIGHT. SO AS I MENTION, THE WAY THAT WE WILL DEAL, THE WAY THAT WE WILL APPROACH THE DEMOS, RATHER THAN BE KIND OF TECHNOLOGY OR PRODUCT FOCUSED WE WILL INTERACT WITH THE SYSTEM AND ASSUME DIFFERENT PERSONAS. NOT PERSONALITIES, BUT THE PERSONAS. SO WE WILL START OFF WITH THE DECISION MAKER WHO IN THIS CASE IS A SALES MANAGER, DYNAMITE 365 WHO MAKES A DISCOVERY , HAS A DATA INSIGHT AND BEING A SALES MANAGER, HE SHARES IT WITH THE WHOLE COMPANY AND THEN IT STARTS THIS CHAIN REACTION AND ALL OF THESE OTHER PERSONAS IN THE ORGANIZATION PARTICIPATE IN THE KIND OF STORY OF THIS DATA INSIGHT. OKAY. SO THANK YOU VERY MUCH JUSTYNA. I’M GOING TO NOW IMPERSONATE THE, THE SALES MANAGER. SO THIS IS GOING TO BE THE QUICKEST DEMO. AND THEN WE WILL MOVE ON TO THE OTHER STUFF LIKE CW, DATABRICKS AND POWER BI AND AZURE ML. SO I’M HERE IN THE 365 HOME PAGE. AND I HAVE HEARD ABOUT THIS SALES INSIGHTS APP. SO I WANT TO TRY THE PREVIEW. AND I HAVE ONE LOADED HERE. SO IMMEDIATELY, IT IS LOADED WITH ALL OF THE DATA FOR MY ORGANIZATION. I CAN SEE WHETHER I’M ON TARGET TO MEET THE QUOTA FOR THE QUARTER. I CAN SEE THE PERFORMANCE OF INDIVIDUAL SALES REPS. I CAN INTERACT WITH THEM BY LEAVING A NOTE. I CAN THEN GO TO THIS BUSINESS TAB WHERE I AM LOADED WITH A BUNCH OF INSIGHTS LIKE, YOU KNOW, PREDICTIVE ANALYTICS AND A BUNCH OF AI BUILT INTO HERE LIKE WE HAVE A MACHINE LEARNING MODEL TO RANK OPPORTUNITIES IN THE PIPELINE TO IDENTIFY OPPORTUNITIES THAT ARE, HAVE A HIGH PROBABILITY OF HIGH REVENUE BUT BEING AT RISK OF BEING LOST SO THEREFORE I CAN TAKE ACTION AS SALES MANAGER. AND THEN I CAN HAVE A CONVERSATION WITH THE SYSTEM IN ENGLISH SO I ALREADY, I HAVE ALREADY LOADED ONE HERE. SO I THINK THAT YOU CAN SEE HERE THAT I MISSPELT IT. IT IS STILL UNDERSTOOD ME. I HAVE TYPED IN, SHOW OUR PIPELINE FUNNEL AND IT IS PICKED THE PRE-BUILT VISUALS THAT IT FEELS IS MOST RELEVANT TO THE QUESTION AND IF I ASK A DIFFERENT QUESTION, LIKE SHOW UP FOR OPPORTUNITIES BY REVENUE AND THEN, IF IT DOESN’T ACTUALLY FIND A VISUAL, IT WILL GENERATE ONE ON THE FLY WHICH IS WHAT IT IS DOING NOW. SO IT IS USING, YOU KNOW, THE Q&A TYPE FUNCTIONALITY THAT IS USED BY POWER BI AND MICROSOFT RESEARCH. IT IS GENERATED A VISUAL THERE FOR ME. OKAY. SO THEN NOW AS THE, DO YOU WANT TO SWITCH? THANK YOU. SO THEN, AS A SALES MANAGER, I GO AROUND TELLING THE WHOLE COMPANY HOW AMAZING THESE INSIGHTS ARE. AND THEN MY FRIEND JUSTYNA WHO IS THE BUSINESS ANALYST WANTS TO VALIDATE WHAT I AM ACTUALLY TALKING ABOUT IS REALLY TRUE. SO SHE IS GOING TO DIG TO THE NEXT LEVEL OF DETAIL IN TERMS OF THE SALES INSIGHTS. >>AWESOME. THANK YOU, CHRISTIAN OVER HERE, WE HAVE GONE THROUGH THE PRECISION MAKER PERSONA. THE ONE THING TO HIGHLIGHT, WE COULD HAVE STARTED AT FLOW AT ANY OF THE BOXES. WHILE STARTING AT THE VERY HIGH LEVEL WITH, WITH DECISION MAKER, BEING ABLE TO SPIN UP THIS APPLICATION AND GETTING INSIGHTS IN LITERALLY SECONDS. WE COULD START WITH THE DATA FROM THE BUSINESS ANALYST PERSPECTIVE OR DATA ENGINEER. SO REALLY, YOU KNOW, THIS PROCESS IS, YOU KNOW, THE MOMENT THAT KIND OF MAKE SENSE TO GO THROUGH THESE BOXES. YOU COULD BE JUMPING AND INTERJECTING AT ANYPOINT. MY CASE, CHRISTIAN HAS GONE AROUND, TOLD THE WHOLE COMPANY, A BUNCH OF NEW WORK TO DO, PULL IN THE DATA FROM DYNAMICS, NO WORRIES. START BUILDING, BASICALLY I WANT TO START BUILDING CUSTOM REPORTS. BECAUSE WE WANT TO GET MORE INSIGHT. I WANT TO GO AHEAD AND CUSTOMIZE THESE REPORTS. BRING IN SOME CUSTOM LOGIC. CUSTOM METRICS AND WE WANT TO BUILD THESE REPORTS DIRECTLY INSIDE POWER BI. SO I AM GOING TO JUMP INTO MY MACHINE OVER HERE. THE FIRST THING THAT I NEED TO DO IS PULL THE DATA INTO POWER BI TO START DOING MY DATA PREPARATION. SO OUT OF CURIOSITY, BEFORE I START, HOW MANY OF YOU USED POWER BI BEFORE? MANY USED DATA FLOWS? OKAY. SO NOT AS MANY. SO HOW MANY OF YOU HAVE HEARD OF DATA FLOWS AT ALL? OKAY. SO DATA FLOWS, IF YOU ARE FAMILIAR WITH POWER BI AND POWER QUERY IN TERMS OF HOW YOU PREPARE YOUR DATA, DATA FLOWS IS ESSENTIALLY THE DATA FLIES VERSION OF THAT. WHERE YOU DECOUPLING YOUR DATA PREPARATION AWAY FROM REPORTING LAYER. IF YOU KNOW, I’M STARTING OUT IN POWER BI SERVICE. I GOT MY DASHBOARD TO MY REPORTS. AND I HAVE THIS NEW TAB OVER HERE CALLED DATA FLOWS. AND WHAT I AM GOING TO DO FIRST OFF IS JUST CREATE A DATA FLOW FROM SCRATCH END TO END. A VERY SIMPLE DATA FLOW AND PULL IN DYNAMICS DATA AND I’M GOING TO SHOW YOU WHAT ELSE YOU CAN DO WITH DATA FLOWS INSIDE POWER BI. I’M GOING TO CREATE A NEW DATA FLOW HERE. CREATED MYSELF, DASHBOARD REPORT, SELECT DATA FLOWS AND I’M GOING TO DEFINE A NEW ENTITY. WE ARE GOING TO TALK ABOUT WHAT LINK ENTITIES AND TALK ABOUT NEW ENTITIES AS WELL. AS YOU CAN SEE, A TON OF DIFFERENT CONNECTORS THAT WE CAN USE. IN THIS PARTICULAR CASE, I WANT TO GO AHEAD AND SELECT MY COMMON DATA SERVICE OVER HERE. AND I’M GOING TO PROVIDE IT WITH MY DYNAMICS URL OVER HERE. I’M GOING TO CONNECT TO DYNAMICS AND I’M GOING TO SIGN IN AS A DIFFERENT USER. AND I’M GOING TO JUST COPY, LET’S SEE, IF I ALREADY GOT SIGNED IN. YEP, I’M HERE. OKAY. AND I’M GOING TO PROVIDE MY PASSWORD. SIGN IN . AND HOPEFULLY I AM NOT GOING TO MISSPELL THAT. AWESOME. AND THEN I’M GOING TO SELECT NEXT. AND THEN THE NEXT THING THAT I CAN DO OF COURSE IS SELECT, YOU KNOW, WHICH ENTITIES I WANT TO GO AHEAD AND PULL OUT FOR MY DYNAMIC INSTANCE. WE SAW, YOU KNOW, CHRISTIAN WAS LOOKING AT A SALES INSIGHT APP. SO WE WANT TO PROBABLY HAVE SOME THINGS AROUND OUR ACCOUNT. WE WANT TO HAVE SOME THINGS ABOUT OUR LEADS OVER HERE. AND I’M GOING TO PULL IN A COUPLE BECAUSE HERE I WANT TO DEMONSTRATE THE END TO END PROCESS OF CREATING A DATA FLOW. WE ARE NOT USING THE DATA FLOW FOR THE REST OF THE DEMO. IT DOESN’T MATTER IF WE HAVE JUST A COUPLE OF THEM HERE. WE WILL PULL IN THREE ENTITIES AS I HAVE DONE HERE. I WILL GO INTO THE NEXT SCREEN. WHAT YOU NOTICE AS I AM GOING THROUGH THIS PROCESS, IF YOU USED POWER QUERY OR POWER BI BEFORE, THIS SHOULD LOOK FAMILIAR. WE ARE BASICALLY EMBEDDED TO POWER QUERY EXPERIENCE THAT IS DIRECTLY INSIDE THE WEB TO MAKE SURE THAT THIS TRANSFORMATION ETL PROCESS THAT ANALYST CAN DO, AS PART OF DATA FLOWS IS A VERY FAMILIAR EXPERIENCE FOR THEM. SO YOU KNOW, IF I WANT TO GO AHEAD AND LET’S SEE, YOU KNOW, CHANGE PARTICULAR DATE. MAYBE CHANGE THIS FROM A DATE TO DATE TO TIME. AND AGAIN, YOU KNOW, THESE ARE FICTITIOUS, YOU KNOW, STEPS THAT I AM DOING. I WANT TO HIGHLIGHT, YOU KNOW, WE HAVE OUR APPLIED STEPS. SO YOU KNOW, EVERYTHING WILL REFRESH IN THAT PARTICULAR ORDER. IF I’M HAPPY WITH THE DATA TRANSFORMATION THAT I HAVE DONE, LET’S GO AHEAD AND SAVE AND CLOSE THIS. THIS WILL TAKE A COUPLE OF SECONDS TO VALIDATE THE QUERIES. THIS WILL BASICALLY HOW I WILL CREATE ENTITIES WITHIN MY DATA FLOW. IF I WANTED TO, I CAN PULL IN FROM OTHER DATA SOURCES TO BASICALLY AUGMENT MY DATA FLOW BUT IN THIS CASE, I’M GOING TO CALL THIS MY SERUM OPPORTUNITIES. AND I’M GOING TO SAVE IT OVER HERE. SO YOU KNOW THIS IS BASICALLY HOW YOU GO ABOUT CREATING A DATA FLOW. THIS IS A VERY, VERY SIMPLE DATA FLOW. PULLED IN DATA FROM DYNAMICS AN SAVED IT. IF WE JUST JUMP INTO THIS NEXT TAB OVER HERE, YOU CAN SEE THE DATA FLOWS CAN GET MORE COMPLICATED. I WILL ZOOM INTO THIS BECAUSE IT IS HARD TO SEE. BASICALLY WHAT I HAVE DONE HERE, EACH OF THESE BOXES THAT YOU SEE WITH THIS LITTLE ICON IS A DATA FLOW. SO I HAVE FOUR DATA FLOWS HERE IN TOTAL. AND I’M INGESTING DATA FROM LOTS OF DIFFERENT SOURCE, EVEN MULTIPLE SOURCES WITHIN ONE DATA FLOW. AND I AM BASICALLY CREATING THESE, I LIKE TO THINK OF DATA FLOWS AND REUSABLE, REPEATABLE PROCESSES THAT EXIST INSIDE POWER BI. AGAIN YOU CAN SEE NOTHING TO DO WITH MY REPORTING HERE. I’M JUST PREPARING MY DATA RIGHT NOW. I CAN BASICALLY, AS I CREATE THE DATA FLOWS YOU WILL SEE, I CAN THEN GO AHEAD AND CONNECT MULTIPLE DIFFERENT REPORTS TO THE SAME DATA FLOW AND CREATE A LOT OF DIFFERENT REPORTS ON TOP OF IT. WHAT I CAN ALSO DO, I CAN LINK ENTITIES TOGETHER. SO YOU WILL NOTICE THESE LITTLE LINES OVER HERE. WHERE I HAVE BASICALLY GOT THE LITTLE NUMBERS IN HERE, SEEING HOW MANY ENTITIES I AM PULLING FROM ONE DATA FLOW TO THE NEXT. I CAN ESSENTIALLY CONSOLIDATE MY DIFFERENT, YOU KNOW, SILO DATA TOGETHER TO A UNIFIED DATA FLOW. SO HERE, YOU KNOW, I AM GOING AHEAD AND BRINGING IN MY FISCAL TARGET, MY SERUM OPPORTUNITY, SALES INVOICES TO GET A HOLISTIC VIEW OF WHAT MY PIPELINE VERSUS GOALS AND ACTUALS LOOK LIKE. AND YOU WILL NOTICE THAT THESE LINES ARE OVER HERE, THEY ARE NOT ACTUALLY, I DIDN’T GO AHEAD AND OFFER THEM, DRAW THE LINES. POWER BI DID THAT FOR ME. POWER BI UNDERSTANDS ALL OF THE FUNCTIONAL DEPENDENCIES. AGAIN, WE WANT TO MAKE THIS THROUGH THE DATA ANALYSTS REALLY SIMPLE. WE DON’T WANT TO HAVE THEM WORRY ABOUT ORCHESTRATION AND OVERWRITING DATA IF THEY ARE BRINGING DATA FROM DIFFERENT PLACES. DIFFERENT SCHEDULES IN PLACE. WE HANDLE ALL OF THAT FOR YOU. WE INSURE THIS AS YOU BUILD THESE, USE THESE DATA FLOWS AS BUILDING BLOCKS TO BUILD ON TOP OF EACH OTHER, YOU WILL NEVER OVERWRITE DATA AND DOING ANYTHING IN THAT KIND OF, THAT MIGHT BE DETRIMENTAL TO YOUR WORK. SO YOU KNOW THE OTHER IMPORTANT THING WHICH WE BOTH MENTIONED DATA FLOWS ARE BUILT ON TOP OF THE AZURE DATA LINK. SO AS AN ANALYST, I AM JUST PULLING IN DATA USING POWER, USING TOOLS I’M FAMILIAR WITH. I AM ACTUALLY DOING DATA PREPARATION ON TOP OF THE DATA LINK. NOW POWER BI HAS THE BUILT-IN DATA LINK THAT WILL USE OUT OF THE BOX. BUT WHERE THINGS START TO BECOME REALLY, REALLY POWERFUL, IT IS WHEN WE START ACTUALLY PLUGGING IN OUR OWN AZURE DATA LINK BEHIND THE SCENES. AS TENANT ADMIN, AS WORK SPACE ADMIN I CAN CHANGE THE STORAGE TYPE TO ACTUALLY USE DATA LINK THAT HAS BEEN CREATED INSIDE OUR OWN AZURE SUBSCRIPTION. AND YOU WILL SEE SOMETHING COOL HERE. WE WILL SWITCH IT INTO THE LIST VIEW. WE WILL POP UP THE AZURE STORAGE EXPLORER OVER HERE. WE ARE GOING TO ZOOM INTO THIS A LITTLE BIT TO MAKE IT EASIER TO SEE. LET’S FIRST TAKE A LOOK HERE, YOU CAN SEE THAT I AM ACTUALLY CONNECTED TO THIS. GONE AHEAD AND BROUGHT IN OVER HERE. PUT MY DIFFERENT BLOB CONTAINERS. ACTUALLY JUST A POWER BI BLOB CONTAINER THAT IS PRESENT INSIDE MY DATA LINK. SO WHAT WE SEE ON THE RIGHT HAND SIDE IS ACTUALLY ALL OF THE WORK SPACES THAT ARE BEING USED TO SAVE DATA ON IN THE DATA LINK IN POWER BI. SO I AM GOING TO SCROLL DOWN. AND FIND MY SALES INSIGHTS WORK SPACE. AND THAT WILL BE A LITTLE BIT HIGHER UP. AND YOU WILL SEE THAT THIS IS OUR SALES INSIGHT WORK SPACE. THE SAME DATA WE HAVE INSIDE THE POWER BI DATA FLOWS IS PRESENT THE CDM FOLDERS INSIDE THE DATA LINK OVER HERE. AND THIS IS WHERE IT STARTS TO GET REALLY, REALLY POWERFUL. BECAUSE POWER BI CAN READ AND WRITE THE CDM FOLDERS. AS YOU CAN SEE THROUGHOUT THE DEMO, AZURE DATA FACTORY. AS CAN AZURE CATTA BRICKS AND NOTEBOOKS. WHAT WE CAN START DOING IS CONSOLIDATED ALL OF THE DATA INTO ONE PLACE. IF I AS ANALYST HAVE INGESTED DATA FROM DYNAMIC, PREPARED IT, DATA FLOW, I CAN NOW COLLABORATE AS YOU SEE WITH MY DATA ENGINEER. I CAN COLLABORATE WITH MY CITIZEN APP DEVELOPER, BIGGEST POWERAPPS READS AND WRITES DATA FLOWS AND EVEN THIRD PARTIES CAN GO AHEAD AND WRITE THESE DATA FLOWS AS WELL. IT IS AN OPEN FORMAT. WE CAN ALL COLLABORATE ON TOP OF THE SAME DATA AND MINIMIZE DATA REDUNDANCIES VERY, VERY EASILY. LET’S LOOK AT WHAT, BASICALLY A CDM FOLDER LOOKS LIKE INSIDE. I AM GOING TO JUMP INTO MY SERUM OPPORTUNITIES THAT YOU SAW ME BRING IN EARLIER. AND YOU WILL SEE OVER HERE, WE HAVE FOLDER FORCE ALL OF THE DIFFERENT ENTITIES THAT WE HAVE. WE HAVE ACCOUNTS, OPPORTUNITIES, AND THESE ARE JUST SAVED CSV EXTRACTS. IT JUST CSV FILES THAT ARE SAVED IN ALL OF THE DIFFERENT ENTITIES. THE REAL KIND OF GLUE THAT BRINGS ALL OF THE THINGS TOGETHER IS THIS MODEL. JSON FILE OVER HERE. THAT ESSENTIALLY PROVIDES ALL OF THE META DATA DEFINITIONS AND THE TRANSFORMATION THAT HAPPENS ESSENTIALLY INSIDE THE DATA FLOWS. I WILL DOWN THIS HERE TO GIVE YOU A SNEAK PEEK AT WHAT IT LOOKS LIKE. I’LL SAVE IT IN A FOALER AND OPEN THIS UP. I HAVE SAVED THIS HERE. WE WILL EDIT IT WITH NOTE PAD. NOPE. YOU CAN BASICALLY SEE OVER HERE, WE HAVE OUR MODEL JSON FILE. JSON DESCRIPTOR. AND WE GO AHEAD AND TAKE A LOOK FOR EXAMPLE OVER HERE, WE HAVE ACCOUNT ENTITY, WE HAVE ALL OF THE ATTRIBUTES OF THE ACCOUNT ENTERSTY, ALL OF THE TRANSFORMATION SAVED. THIS IS REALLY ESSENTIALLY THE GLUE BETWEEN ALL OF THE SERVICES. THIS IS THE FILE THAT POWER BI CAN READ AND WRITE. SO CAN, YOU KNOW, ALL OF THE DIFFERENT AZURE DATA SERVICES. AS YOU SEE LATER ON, LET’S SAY WE USE SOMETHING LIKE DATABRICKS, TO MAYBE DO SOME ENRICHMENT ON TOP OF THE DATA AND WRITE A NEW CDM FOLDER. POWER BI CAN READ THAT AS EXTERNAL DATA FLOW. AS ANALYST, TO ME, I’M JUST READING ANOTHER DATA FLOW INTO POWER BI THAT HAPPENS TO BE COMING FROM A DIFFERENT WORK SPACE. WHAT WE REALLY WANT TO DO IS INSURE THAT THE ANALYST, THE DATA SCIENTIST, THE DATA ENGINEER, THE CITIZEN DEVELOPER CAN NOW COLLABORATE TOGETHER ON THE SAME DATA LEVERAGING THEIR OWN TOOL SO THEY DON’T HAVE TO CROSS OVER INTO EACH OTHER’S TOOLS. WORK ON TOP OF THE OWN TOOLS. COLLABORATE ON TOP OF THE SAME DATA. LET’S NOT JUMP INTO POWER BI FOR A SECOND. SO I HAVE GONE AHEAD, PREPARED ALL OF THE DATA. AND NOW LET’S TAKE A LOOK AT WHAT THE REPORTING LOOKS LIKE. SO I COULD GO AHEAD AND IF I WANTED TO START FROM SCRATCH, GET DATA. AND YOU WILL NOTICE THAT WE HAVE THIS NEW CONNECTOR INSIDE POWER BI. WE HAVE POWER BI SETS AND ALSO HAVE POWER BI DATA FLOWS. I CAN CONNECT TO POWER BI DATA FLOWS AN CREATE MULTIPLE DIFFERENT REPORTS ON TOP OF IT. I’M NOT GOING TO DO THIS FROM SCRATCH JUST TO SAVE TIME. BUT I WANTED TO SHOW YOU THAT WE HAVE ALL OF THE DIFFERENT WORK SPACES OVER HERE. WE HAVE SALES INSIGHT WORK SPACE AT THE BOTTOM TO CONNECT THAT, BRING IN MY ENTITIES AND START CREATING MY REPORTS. NOW YOU KNOW I HAVE GONE AHEAD AND GOT THEM STARTED INSIDE POWER BI OVER HERE. YOU KNOW, CHRISTIAN WANTED SOME CUSTOM REPORTS. GONE AHEAD. STARTED BUILDING THEM OUT. YOU KNOW, YOU CAN USE POWER BI TO LET’S SAY FIND INSIGHTS QUICKLY IN DATA. HERE, FOR EXAMPLE, WE HAVE PERFORMANCE ACROSS QUARTERS. YOU CAN SEE, INCREASE IN Q4. IF I WANT TO ANALYZE THIS, EXPLAIN THE INCREASE, POWER BI RUN ANALYTIC FORCE ME. I CAN SEE THE INCREASE IS MOSTLY COMING FROM ONE OF MY SALES PEOPLE. ANGELO OVER HERE HAS ACTUALLY HAD AN INCREASE IN ACTUAL DEAL SIZE. THAT IS INTERESTING. MAYBE CHRISTIAN WILL WANT TO KNOW THAT. I CAN ADD THAT TO MY REPORT. ADD IT TO MY POWER BI. IT SOUNDS LIKE YOU ARE FAMILIAR WITH POWER BIS AND I’M NOT GOING TO GO INTO HOW YOU OFFER DIFFERENT VISUALS INSIDE HERE. VERY QUICKLY, YOU START CREATING CUSTOM REPORTS AN CUSTOM GPI’S AND CUSTOM METRICS AN SO ON. NOW I WANT TO ACTUALLY MAYBE DO MORE OF THIS, CHRISTIAN THE BUSINESS USER HAS BEEN DEMANDING MOREND BRING THIS MORE DATA TOGETHER MUCH WE ARE GOING TO ACTUALLY FLIP OVER AND TAKE A LOOK AT WHAT THE DATA ENGINEER CAN DO WITH SOME OF THE DATA AND DATA INSIDE THE COMMON FOLDERS.>>THANK YOU VERY MUCH, JUSTYNA. THAT WAS VERY, VERY COOL. SO YES, SO WHAT JUSTYNA DID IN THE ORGANIZATION HAS TAKEN OFF AND NOW IT IS, HAS BEEN ASKED TO TAKE THIS DYNAMIC 365 DATA AND OPERATIONALIZE THE ETL FOR IT. AND INTEGRATE THAT DATA INTO THE CORPORATE, WAREHOUSE FOR REUSABILITY THROUGHOUT THE ORGANIZATION. ALL RIGHT. SO I AM NOW GOING TO SWITCH HATS TO THE DATA ENGINEER. WHO IS GOING TO LEVERAGE THE DATA THAT JUSTYNA EXPOSED IN THE ORGANIZATION. AND IS NOW IN THE DATA LINK. OKAY. THANK YOU VERY MUCH. OKAY. SO I AM NOW USING AZURE DATA FACTORY WHICH IS, WHO HERE KNOWS OF AZURE DATA FACTORY? OKAY. THE MAJORITY OF YOU. SO YOU WILL KNOW THAT IT IS AN ETL SERVICE TO CREATE, MONITOR AND SCHEDULE DATA PIPELINES. SO THERE IS SOME KIND OF OVERLAPPING TERMS OF THE FUNCTIONALITY BETWEEN AZURE DATA FACTORY AND POWER BI DATA FLOWS. BUT THE PERSONA IS DIFFERENT. THE DATA FLOWS WAS TARGETED AT THE BUSINESS ANALYST WHERE THIS IS MORE FOR OPERATIONALIZING THE ETL FOR, YOU KNOW, AND SUPPORTS THE CROSS ENVIRONMENTS AN CHECK INTO CONTROL AND LIFE-CYCLE MANAGEMENT. THE PERSONA IN THIS CASE, DATA ENGINEER THAT WORKS IN IT. ALL RIGHT. SO I’M GOING TO TAKE THE DYNAMICS 365 PRODUCT DATA THAT JUSTYNA EXPOSED IN THE DATA LINK, I’M GOING TO READ IT. USING THE COMMON DATA FORMAT. I’M ALSO GOING TO TAKE SOME DATA FROM SAP. AND I’M GOING TO PUT IT IN THE DATA LINK AS WELL. AGAIN IT IS GOING TO BE STORED USING THE COMMON DATA MODEL FORMAT AND THEN USE AZURE DATABRICKS TO PERFORM A MERGE OPERATION. SO HERE WE HAVE A DATABRICKS NOTEBOOK THAT IS PRIMARY BY THE FOLDERS IN ADLS. THE DATABRICKS NOTEBOOK, IF YOU SEE HERE, CREATING POINTERS TO THE MODEL. JSON FILES. ALL OF THE SCHEMER INFORMATION. AND WE THEN WILL READ AND, READ THE DYNAMICS DATA AND THE SAP DATA. SO AS YOU CAN SEE HERE, WE ARE JUST READING THE CDM FILE AND WE ARE DYNAMICALLY INFERRING THE SCHEMER. ALL RIGHT. SO DATABRICKS HAS A KIND OF NATIVE CONNECTOR FOR THE CDM ENTITIES. AND THEN I WILL USE SPARK SQL TO ACTUALLY PERFORM THE MERGE OPERATION POTENTIALLY BASED OFF OF SOME DATA MANAGEMENT KEY. THEN I’M GOING TO WRITE A NEW VERSION OF PRODUCT. THE GOLDEN VERSION OF PRODUCT AS IT WERE TO A NEW CDM ENTITY. ALL RIGHT. THAT WILL GENERATE A NEW MODEL. JSON FILE. AND THIS IS GOING TO HAVE THE ATTRIBUTES ACROSS BOTH OF THE SOURCE SYSTEMS IN THE SINGLE INTEGRATED VERSION OF PRODUCT. I WILL THEN LOAD THIS INTO THE DATA WAREHOUSE. AND THIS HAPPENS TO BE AN ABSOLUTELY HUGE DATA WAREHOUSE. WHO WAS HERE FOR SCOTT GUTHRIE’S KEYNOTE? ANYONE? YOU MAY HAVE SEEN THIS DATA WAREHOUSE THERE. IT IS AN ABSOLUTE BEAST OF A DATA WAREHOUSE. IT IS 30, 000 DWU. THIS IS OVER 4, 000 CAUSES, CAUSE 20 TERABYTES OF MEMORY. THIS IS A PETABYTE OF DATA. AND THEY LET US HAVE, YOU KNOW, HAVE ACCESS TO IT. BECAUSE YOU KNOW WE ARE POWER BI AND DEAL IN PETABYTES ALL OF THE TIME. SERIOUSLY, WE CAN INTERACTIVELY WRITE REPORTS ON TOP OF PETABYTES DATASETS. IN FACT, WHAT I HAVE HERE, LET ME JUST HIDE THESE. AND I’M GOING TO GO WITH THIS LINE ITEM TABLE IS THE BIG ONE THAT HAS THE MOST ROWS IN IT. IF I FIND THIS MEASURE, AND YOU
CAN SEE RIGHT HERE, THIS IS SIMPLY THE COUNT OF THE ROWS AND THE TABLE. AND I’M GOING TO CASUALLY DRAG THIS ON TO THE CANVAS. IF YOU SEE SCOTT GUTHRIE’S KEYNOTE, THEY STOLE MY THUNDER. I SAY, HOW MANY ROWS DO YOU THINK ARE IN HERE? HUNDRED MILLION? A BILLION? THIS IS LOW AND BEHOLD 6 TRILLION ROWS. PETABYTE OF DATA. AS YOU CAN SEE, I’M GETTING INSTANT RESPONSE TIMES OVER PETABYTE OF DATA. AS WE KNOW THE TECHNOLOGY UP UNTIL PRETTY RECENTLY, THIS SHOULD BE PHYSICALLY IMPOSSIBLE. RIGHT? I’M GETTING INSTANT RESPONSE TIMES OVER A PETABYTE OF DATA. I’M GOING TO BREAK OUT ACCOUNT BY, LET ME FIND DATE HERE. I’M GOING TO MAKE IT A BAR CHART. I’M GOING TO BREAK IT OUT BY THE DIVISION. AND I’M GOING TO MAKE IT NICE AND BIG. I’M GOING TO GO AHEAD AND, WHOOPS, I’M GOING TO FILTER IT BY JUST THE SHIPMENTS DELAYED. NOW I’M GOING TO USE THIS CROSS FILTERING FEATURE IN POWER BI WHERE I CLICK ON ANY RANDOM PART OF THE REPORT AND IT WILL INTELLIGENTLY CROSS FILTER OTHER VISUALS. I’M NOW GOING TO FIND OUT WHICH OF THE DISTRIBUTION CENTERS FOR THE BEAUTY AND PERSONAL CARE DELAYED SHIPMENTS ON THIS DATE WHERE WE HAD A SPIKE AND NOW I WANT TO DRILL THROUGH TO THE DETAILS FOR THIS PARTICULAR DISTRIBUTION CENTER. SO THE REASON THAT THIS HAS BEEN SUPER FAST IS THAT IT HAS BEEN HIT IN THE MEMORY AGGREGATED CASH IN POWER BI. WE CREATE AN AGGRAVATED CASH WHICH IS VERY SMALL IN TERMS OF MEMORY CONSUMPTION, RIGHT? OVER THIS MASSIVE DATA SET. AND THEN IF WE DRILL THROUGH TO THE DETAIL LEVEL, WHERE THERE IS NO CASH, IT WILL THEN JUST ON THE FLY SUBMIT A DIRECT QUERY TO SQL DATA WAREHOUSE. THEN IT CAME BACK REALLY FAST BECAUSE SQL DATA WAREHOUSE CAN OPTIMIZE FOR THE TARGETED QUERIES PRETTY EASILY. IF YOU KNOW WHERE IT WILL LAND, INDEX FOR IT AND EVERYTHING. WHERE AGGREGATED LEVEL, WHICH COULD BE TOUCHING MANY DIFFERENT TABLES AN LOTS AND LOTS OF DIFFERENT DATA, YOU HIT THE POWER BASE CASH, YOU HIT THERE WITH A TINY BIT OF MEMORY CONSUMPTION AND THEN THE QUERIES THAT DO GET THROUGH TO THE DATA WAREHOUSE, ING YOU PROTECT THE CURRENCY LIMITS IN YOUR WAREHOUSE. THEN THE QUERIES THAT GET THROUGH ARE MORE TARGET AND EASIER TO OPTIMIZE FOR. AND END UP WITH BALANCED ARCHITECTURE. I’LL GO INTO THIS IN TWO HOURS IN MY SESSION. THIS IS AGGREGATED SESSION IN POWER BI THAT UNLOCK THE DATASETS AND PETABYTE SCALE AND DATABRICKS AN ANY DIRECT QUERY SOURCES. SO WE NOW INTEGRATED THIS INTO THE DATA WAREHOUSE. WE HAVE GOT A HUGE DATA WAREHOUSE. THE BUSINESS IS USING IT. IT IS DRIVING THE DECISIONS IN A CONSISTENT WAY THROUGHOUT THE DATA WAREHOUSE. SORRY, THROUGHOUT THE ORGANIZATION. AND NOW IT IS, IT HAS GONE OVER TO THE AI GROUP. AND THEY HAVE GOT WIND OF IT. THEY WANT A PIECE OF THE ACTION AS WELL. >>COOL. THANK YOU, CHRISTIAN. YES. SO YOU KNOW OUR BUSINESS USER NOW HAS THESE CUSTOM REPORTS IN POWER BI. THEY HAVE REPORTS FOR SHIPMENT DELAYS ON TOP OF THEIR SQL CW BUT WANT MORE. THEY WANT REPORTS THAT HAVE MORE MACHINE LEARNING BUILT INTO THEM TOO. NOW WHAT WE CAN DO, OF COURSE, AS YOU CAN SEE, THE APP HAD BUILT-IN MODELS AND MAYBE YOU WANT TO CUSTOMIZE MODELS. YOU WANT YOUR DATA SCIENTIST TO BUILD THEIR OWN MACHINE LEARNING MODELS USING THE AZURE DATA SERVICES. NOT A PROBLEM. WHAT WE ARE GOING TO DO, START HERE INSIDE THE AZURE NOTEBOOKS. AND I’M GOING TO FLIP OVER TO THE SCREEN. AND SO YOU KNOW AZURE NOTEBOOKS IS A PLACE WHERE YOU CAN GO AHEAD AND CREATE JUPITER NOTEBOOKS ON TOP OF AZURE TO WRITE, LET’S SAY, IN THIS CASE, WRITING PYTHON CODE TO CREATE OURSELVES A MACHINE LEARNING EXPERIMENT. AND WE ARE GOING BE USING THE TECHNOLOGY THAT CHRISTIAN MENTIONED EARLIER CALLED AUTOMATED AMOUNT. SO AUTOMATED ML IS BUILT ON TOP OF AZURE MACHINE LEARNING. MACHINE LEARNING TOOLKIT FOR DATA SCIENTISTS. BUT AUTOMATED ML, ESSENTIALLY AUTOMATES A BUNCH OF THE PROCESSES FOR YOU. AS YOU SEE OVER HERE, WHAT IT ESSENTIALLY DOES, GIVES YOU PARAMETERS AND ESSENTIALLY DO FEATURE SELECTION FOR YOU. ALGORITHM SELECTION, HYPER PARAMETER TUNE, ITERATE THROUGH LOTS AND LOTS OF DIFFERENT MODEL FORCE YOU. AND CREATE THE BEST POSSIBLE MODEL THAT CAN, AND THAT IS THE ONE THAT WILL BASICALLY GET RETURNED. BEFORE THAT, WE ARE GETTING A LITTLE BIT AHEAD OF OURSELVES. LET’S LOOK AT NOTEBOOK OVER HERE. AND THERE IS SOMETHING THAT I WANTED TO DRAW YOUR ATTENTION TO RIGHT OVER HERE. WHICH IS THE FACT THAT, YOU KNOW, WE ARE BASICALLY AGAIN, READING IN OUR MODEL. JSON FILE. AGAIN, IF YOU WANT TO USE AZURE NOTEBOOK, AND YOU WANT TO HAVE SOME DATA IN THE DATA LINK THAT HAS BEEN PREPARED AND YOU WANT TO READ IT IN, NOT A PROBLEM. YOU CAN READ IT IN, IN THE SAME FORMAT. READ IT IN THE MODEL. JSON FILE. UNDERSTAND. CONVERTED IT TO A DATA FRAME AND SEEING AND VISUALIZING THE OPPORTUNITIES ENTITY THAT WE BROUGHT IN EARLIER. THEN WE ARE GOING TO DO TRANSFORMATION USING PYTHON. GO AHEAD AND CLEAN THE DATA UP A LITTLE BIT. JUST REDUCE IT DOWN TO THE FEATURES THAT WE WANT TO USE FOR THE MACHINE LEARNING MODEL. AND THEN THIS IS WHERE WE ACTUALLY CONFIGURE AUTOMATED AMOUNT. YOU WILL SEE OVER HERE, WE NEED TO GIVE IT A NUMBER OF DIFFERENT PARAMETERS, FOR EXAMPLE, WE ARE GOING TO TELL IT THAT WE WILL CREATE A REGRESSION MODEL. MODEL THAT WILL TELL US WHETHER AN OPPORTUNITY, WHAT IS THE CLOSE PROBABILITY LIKELIHOOD OF AN OPPORTUNITY. BETWEEN ZERO TO A HUNDRED. THEN A LABEL, WHETHER WE THINK THAT IT IS LIKELY TO CLOSE OR NOT. YOU KNOW, YOU CAN GO AHEAD AND OFFER THINGS LIKE, YOU KNOW, WHAT IS YOUR PRIMARY METRIC THAT YOU ARE TRYING TO OPTIMIZE FOR. COST FUNCTION . AND IN THIS CASE, ROOT SQUARE MEAN ERROR. HOW MANY ITERATIONS YOU WANT TO RUN? BLACK LIST ANY MODEL? YOU HAVE A LOT OF GRANULAR CONTROL. BEHIND THE SCENE, AUTOMATED ML WILL BASICALLY RUN AND GO AHEAD AND DO A LOT OF DIFFERENT, OOPS, I DIDN’T WANT TO DO THAT. ITERATE THROUGH A LOT OF DIFFERENT MODELS AND ALGORITHMS WITH DIFFERENT PARAMETER TUNINGS. BASICALLY RETRIEVE THE BEST MODEL THAT WE WENT AHEAD AND FOUND. THEN GOING AHEAD AND CREATE A SCORING SCRIPT. ONCE HAPPY WITH THE MODEL WE CAN BASICALLY DEPLOY IT. THAT IS WHAT WE ARE GOING TO DO HERE AT THE END WHERE WE ARE BASICALLY GOING TO CONTAIN CONTAINER IMAGE. AZURE. AS A DATA SCIENTIST I CAN COLLABORATE WITH THE ANALYST AND WORK ON TOP OF THE SAME DATA BUT I AM GOING TO WORK INSIDE MY OWN TOOLSET WHICH I AM COMFORTABLE WITH. NOT INTO POWER BI MACHINE LEARNING MODELS BUT USE THINGS LIKE JUPITER NOTEBOOK, PYTHON CODE AND I CAN ABSOLUTELY DO THAT. NOW WE HAVE A BIT OF A CHALLENGE BECAUSE WE WANT TO ACTUALLY EXPOSE THIS MODEL DIRECTLY TO THE ANALYST INSIDE POWER BI. AND THAT IS NOT A PROBLEM. WE ARE GOING TO BE ABLE TO DO THAT, TOO. ONCE THIS MODEL IS DEPLOYED YOU CAN SEE IT AS AN AZURE RESOURCE. INSIDE THE AZURE PORTAL. IF YOU WANTED TO TAKE A LITTLE BIT OF THE CLOSER LOOK AT, YOU KNOW, THE ITERATION THAT IT RAN, GO AHEAD AND DO THAT IN A MORE VISUAL WAY AS WELL. LET’S JUMP INTO OPPORTUNITY MODEL HERE. JUMP INTO OUR RUNS. SO YOU CAN ACTUALLY SEE OVER HERE HOW WE WENT AHEAD AND OPTIMIZING, YOU KNOW, OUR ERROR, WHERE WE CAN SEE THAT WE ARE MINIMIZING THAT WITH EVERY NEW MODEL THAT WE CREATE. IMPORTANT THING THAT I WANT TO DO HERE, ACCESS CONTROL AND I AM GOING TO GO AHEAD AND ADD A NEW ROLE ASSIGNMENT. IF I’M THE DATA SCIENTIST AND I WANT TO EXPOSE THESE MODELS THROUGH THE POWER BI REPORTS AN LET’S FOR A SECOND SAY THAT THE DATA SCIENTIST IS NOT THE ANALYST OR NOT THE SAME PERSON, IT WILL BE SOMEONE ELSE. ALL THEY HAVE TO DO IS SPECIFY A ROLE, CONTRIBUTOR, AND THEN THEY HAVE TO BASICALLY FIND THE ANALYST IN THE LIST OVER HERE. AND THEN COULD GO AHEAD AND SHARE THE MODEL. THAT IS EASY AS IT IS. FOR THE DATA SCIENTIST TO COLLABORATE WITH THE ANALYST. WE KNOW THAT WITH DATA SCIENTIST, YOU KNOW, THE MODEL GETS CREATED AND OFTEN, YOU YOU KNOW, IT IS HARD TO SHARE THE MODELS WITH BUSINESS USERS IN CONCEIVABLE AND MEANINGFUL WAY. THIS IS REALLY EASY WAY TO EXPOSE THIS TO POWER BI AND BUILD INTERACTIVE REPORTS ON TOP OF IT. LET’S LOOK AT WHAT THAT WOULD LOOK LIKE. I’LL JUMP BACK INTO THE DATA FLOW. ONE WE STARTED WITH THE AT BEGINNING. LET’S GO AND ZOOM IN A LITTLE BIT OVER HERE. LET’S ZOOM IN SERUM OPPORTUNITIES. I WANT TO ESSENTIALLY ENRICH MY OPPORTUNITY ENTITY WITH MACHINE LEARNING MODEL THAT CREATED AND SHARED WITH ME. I’M GOING TO GO AHEAD AND I’M GOING TO EDIT THIS PARTICULAR DATA FLOW. AND THEN I’M GOING TO JUMP INTO OPPORTUNITIES OVER HERE. AND THEN I’M GOING TO EDIT THIS ENTITY. AND NOW YOU WILL NOTICE THAT WE HAVE THIS NEW BUTTON INSIDE POWER BI. INSIDE THE POWER QUERY EDITOR AND DATA FLOWS CALLED AI INSIGHTS. IF YOU WANT TO LEARN MORE ABOUT THIS, I HAVE A SESSION TOMORROW MORNING LOOKING AT POWER BI AND AI WITH A LOT MORE DETAIL WITH THIS. BASICALLY, IF WE JUMP INTO AI INSIDE, THIS WILL TAKE A SECOND TO LOAD, BECAUSE A LOT OF MACHINE LEARNING MODELS WHAT WE ARE DOING IS TAKING AZURE TECHNOLOGIES AND BUILDING THEM DIRECTLY INTO POWER BI THAT YOU KNOW, BASICALLY OUR ANALYST CAN BENEFIT FROM THEM, TOO. SO TOMORROW, I’M GOING TO GO INTO MORE DETAILS HOW WE INTEGRATED COGNITIVE SERVICES DIRECTLY INTO POWER BI. TODAY I WANT TO SHOW YOU THE FACT THAT YOU CAN SHARE AZURE MACHINE LEARNING MODELS. YOU MIGHT SAY, YEAH, I SEEN IT. BUT WE BUILT AUTOMATED LEARNING MODEL NOT AZURE MACHINE LEARNING MODEL. ONCE DEPLOY SERVICE MODEL, SAME THING. SAME TECHNOLOGY BEHIND THE SCENES. AZURE MACHINE LEARNING WEB SERVICES ARE BASICALLY THE SAME AS AUTOS MATED WEB SERVICES. IN THIS CASE, WE HAVE A LIST OF DIFFERENT MODELS THAT WERE CREATED AND HERE IS MY OPPORTUNITY SCORING MODEL THAT HAS BEEN SHARED WITH ME. AND AS AN ANALYST, ALL I HAVE TO DO NOW IS BASICALLY MAP THIS MODEL TO THE SCHEMA, OR MAP THE DATA TO THE SCHEMA THAT THE MODEL EXPECTS. I’VE GOT THE FIELDS THAT THE MODEL IS EXPECTING. AND YOU KNOW I COULD GO AHEAD AND LOOK THROUGH THE LIST OF THINGS THAT I HAVE INSIGHT THE OPPORTUNITY ENTITY AND START MAPPING THESE ESSENTIALITY TO MY DATA. I’M NOT GOING TO GO AHEAD AND MAP ALL OF THIS RIGHT NOW. FOR THE PURPOSE OF TIME. I’LL SHOW YOU WHAT IT LOOK LIKE ONCE WE ACTUALLY SEE HERE, INVOKE THE AZURE MACHINE LEARNING OPPORTUNITY MODEL AND EXPAND IT OUT. BECAUSE IT IS COLUMN. LET’S JUMP ALL OF THE WAY TO THE RIGHT HAND SIDE OVER HERE. YOU WILL BASICALLY SEE WHAT WE HAVE CREATED IS TWO NEW ADDITIONAL COLUMNS TO MY ENTITY, TO MY OPPORTUNITY ENTRY. WHICH IS MY OPPORTUNITY CLOSE PROBABILITY SCORE. HOW LIKELY TO CLOSE THIS OPPORTUNITY? AND THEN WE HAVE ASSIGNED IT A LABEL BASED ON THE THRESHOLD THAT WE SET. FOR EXAMPLE, IMAGINE WE SAID ANYTHING, SET ANYTHING ABOVE 70 IS A YES AND ANYTHING BELOW IS A NO. YOU KNOW, SOMETHING OF THAT SORT. AND SO ESSENTIALLY NOW, WE HAVE GONE AHEAD AND ENRICHED OUR DATA WITH THE MACHINE LEARNING MODEL THAT THE DATA SCIENTIST HAS CREATED. IF WE JUMP IN ORIGINAL REPORT, NOW START DOING MORE COMPLEX THINGS. WE CAN, FOR EXAMPLE, SURFACE THE MODEL IN SOME OF OUR, YOU KNOW, REPORTING ARTIFACTS. SO HERE FOR EXAMPLE WE HAVE ACTUALLY PLOTTED, PREDICTED OPPORTUNITY CONVERSION. SO IF WE LOOK AT THE Y ACCESS OVER HERE, WE HAVE THE CONFIDENCE OF CLOSING THIS OPPORTUNITY WHICH IS ANYWHERE FROM ZERO TO A HUNDRED. THEN PLOTTED BY THE ESTIMATED REVENUE. HOW MUCH IS THIS OPPORTUNITY WORTH? I CAN START QUICKLY AS A BUSINESS USER FIGURING OUT WHICH OPPORTUNITIES I SHOULD BE FOCUSING ON. HERE, FOR EXAMPLE, WE HAVE AN OPPORTUNITY THAT HAS QUITE HIGH ESTIMATED REVENUE. BUT WE CAN SEE THAT THE CONFIDENCE OF CLOSING IT IS PRETTY LOW. SO I CAN SELECT THE OPPORTUNITY HERE AND CROSS FILTER MY ENTIRE REPORT. I CAN SEE THIS IS BANK PUBLISHING. SEE WHAT THE PIPELINE WITH THEM LOOKS LIKE. OUR DEAL SIZE FOR THEM, LET’S SAY THE PAST QUARTER. MAYBE AS NEXT STEP, FOLLOW WITH THE SALES PERSON AND THEN INSURE THAT THEY ARE FOCUSING ON OPPORTUNITY THAT IS SOMETHING VERY IMPORTANT FOR US. I WANTED TO ALSO SHOW YOU QUICKLY, SLIGHTLY GO OFF ON A TANGENT, BUT YOU KNOW, WE HAVE THE SAME TECHNOLOGY THAT I JUST SHOWED YOU WHICH IS, YOU KNOW, ULTRAMATED MACHINE LEARNING TECHNOLOGY THAT IS BUILT DIRECTLY INTO DIRECTLY POWER BI. IN THE CASE OF DATA SCIENTIST, THEY DO WANT TO USE AZURE NOTEBOOKS TO CREATE THE MACHINE LEARNING MODELS. WHAT IF THE ANALYST WANTED TO BE A LITTLE BIT BRAVE AND START DABBLING IN MACHINE LEARNING AND NOT GO OFF AND VENTURING INTO THE AZURE WORLD. NOW THEY CAN DO THAT WITH DIRECTLY INSIDE POWER BI. SO I AM GOING TO CHANGE THE SCENARIO A LITTLE BIT. I’M GOING TO BE LOOKING AT CUSTOMERS WHO ARE SUBSCRIBING TO MY CLOUD SERVICE AND IN THIS CASE, WE BASICALLY GOT, YOU KNOW, A BASIC REPORT THAT LOOKS AT THE NEW CUSTOMER WHOSE CAME IN, LET’S SAY, IN THE MONTH OF FEBRUARY. ALONG WITH THE INTERESTS, THE ROLES AND REGIONS. IMAGINE USE POWER BI TO ADD MACHINE LEARNING TO THIS. IF I JUMP BACK INTO POWER BI OVER HERE, YOU WILL SEE AGAIN, I’M INSIDE DATA FLOWS. I’M INSIDE THE ENTITIES AND HAVE HISTORICAL CUSTOMER FEEDBACK. IMAGINE WE HAVE GONE AHEAD AND SURVEYED CUSTOMER WHOSE HAVE USED THE SERVICE FOR A COUPLE OF MONTHS. FIGURED OUT WHICH CUSTOMERS ARE HAPPY AND NOT SO HAPPY OF THE SERVICE. YOU NOTICE IN POWER BI, WE HAVE THIS KIND OF BRAIN ICON THAT IS A LIGHTNING BULB. IT SHOWS YOU HOW INTELLIGENT IT IS. BUT BASICALLY THIS IS OUR ICON FOR MACHINE LEARNING IN POWER BI. AND IF I SELECT THIS, WHAT WE CAN DO IS ACTUALLY ADD A MACHINE LEARNING MODEL. NOW IF WE REMEMBER WHAT WE SAW INSIDE THE AZURE NOTEBOOKS, AND I’M GOING TO JUST JUMP BACK INTO THIS FOR A SECOND, YOU KNOW, WE WENT AHEAD AND POPULATED ALL OF THESE PARAMETERS TO JUST, WE WANT TO REGRESSION, YOU KNOW, WE WANT TO RUN THIS MANY ITERATIONS AND SO ON. WE ARE GOING TO USE POWER BI TO ESSENTIALLY DEFINE ALL OF THESE THINGS. YOU KNOW, IN MORE OF A POINT AND CLICK WAY WHICH OUR ANALYSTS ARE AGAIN MORE USED TO WORKING WITH. THE FIRST THING I NEED TO DO, DEFINE WHAT IS THE ENTITY, ACTUALLY, WHAT IS THE COLUMN THAT I WANT TO PREDICT. IN MY CASE, THIS IS WHETHER WE HAVE A HIGH RITTING OR NOT. A BINARY CLASSIFICATION AND HIGH RATING IS BASICALLY A TRUE OR FALSE COLUMN SAYING, YES, WE RECEIVED THE HIGH RATING FROM THE CUSTOMER OR WE DIDN’T. AND THEN WHEN I CLICK NEXT, POWER BI WILL TAKE A COUPLE OF SECONDS TO TAKE A LOOK AT COLUMN THAT I HAVE SELECTED. AND IT IS GOING TO KIND OF TELL US WHICH MODELS IT IS ESSENTIALLY THINKS THAT WE SHOULD BE ABLE TO LEVERAGE OVER HERE. AND THIS IS, YEP. OKAY. SO WE CAN SEE, THESE ARE THE KIND OF MODELS THAT WE CAN CREATE AND BECAUSE I HAVE A BINARY FIELD I WANT TO CREATE A BINARY PREDICTION MODEL. SO THEN THE NEXT THING THAT I AM GOING TO DO IS GOING TO SELECT WHICH DATA DO I WANT TO USE FOR THIS PARTICULAR MODEL? AND AGAIN, THIS IS JUST ME SELECTING MY FEATURES. BEFORE WE DID THAT IN CODE. NOW IN POWER BI. I CAN JUST, IF I DON’T AGREE WITH THE SELECTION OF POWER BI HAS SORT OF RECOMMENDED HERE, I CAN CHANGE THESE AROUND. AND THAT IS PRETTY MUCH IT. THEN I JUST GIVE MY MODEL A NAME. FOR EXAMPLE, CUSTOMER FEEDBACK. I TELL IT WHAT IS A TRUE OUTCOME LABEL MEAN AND WHAT IS A FALSE OUTCOME MEAN. SO TRUE OUTCOME WOULD BE HIGH RISK BASICALLY, AND LET’S SAY FALSE OUTCOME WOULD BELOW RISK IN THIS PARTICULAR CASE. IF I WAS TO SAVE RIGHT NOW, WHAT WE SAW HAPPEN WITH THE AZURE NOTEBOOKS WOULD BASICALLY HAPPEN HERE. WE WOULD RUN THROUGH AND ITERATE THROUGH DIFFERENT MODELS AND FEATURE SELECTION, ALGORITHM SELECTION, HYPER PARAMETER TUNING TO BUILD THE BEST MODEL. THAT WILL TAKE A COUPLE OF MINUTES. NOT LIVE HERE. INSTEAD I’M GOING TO SHOW YOU WHAT YOU WOULD GET INSIDE POWER BI. AND SO HERE IS THE MACHINE LEARNING MODEL THAT I BUILT EARLIER. EXACTLY ON THE SAME TYPE OF DATA. WHAT I CAN DO IS VIEW THIS PERFORMANCE REPORT TO LOOK AT HOW WELL MY MODEL IS PERFORMING. WHAT YOU WILL NOTICE OVER HERE, WE HAVE ACTUALLY BUILT A CUSTOMER REPORT TO HELP EXPLAIN TO THE ANALYSTS WHAT THE MACHINE LEARNING MODEL IS SAYING. WE FOR EXAMPLE HAVE VISUAL FORCE GIVING ME HOW MANY TRUE NEGATIVES VERSUS FALSE NEGATIVES, TRUE POSITIVES VERSUS FALSE POSITIVES AND SO ON. WE GIVE YOU PRECISION, RECALL, ON THE RIGHT HAND SIDE, WE GIVE YOU A DETAILED EXPLANATION OF WHAT ALL OF THESE METRICS ARE SAYING. AGAIN, OUR ANNUAL YISES ARE NOT — ANALYSTS ARE NOT DATA SCIENTISTS. WE WANT TO MAKE SURE THAT WE ARE TRANSPARENT. WE HAVE WHAT SAMPLE AND WHAT KIND OF MODEL WE FIT. YOU CAN GO AHEAD AND TWEAK THINGS LIKE THE PROBABILITY THRESHOLD AND SEE HOW THINGS INTERACTIVELY CHANGE. THE FINAL THING IS SHOW YOU WHAT ELSE WITH AUTOMATED ML INSIDE POWER BI. I’M GOING TO JUMP INTO THE REPORT THAT I SHOWED YOU EARLIER. I’M GOING TO JUMP INTO THIS CUSTOMER RISK PAGE THAT HE HAS. NOW MOVE INTO PREDICTIVE STATISTICS. WE WENT AHEAD AND HAD A RISK SCORE FOR EVERY SINGLE CUSTOMER THAT WE HAD. YOU KNOW, IN THE FIRST MODEL WE ACTUALLY WERE AGGREGATING THAT DATA A LITTLE BIT AND BRINGING IN OTHER THINGS SUCH AS OUR ESTIMATED REVENUE. HERE WE ACTUALLY WANT TO SEE THINGS ROW BY ROW. IF WE LOOK AT, WHICH OF OUR CUSTOMERS, MY CUSTOMERS BASICALLY AT RISK OF GIVING US LOW SURVEY RESPONSES IN THE FUTURE. AND HERE, FOR EXAMPLE, WE CAN SEE LET’S SEE, JAMES AND MEGAN HAVE A HIGH RISK SCORE. WHAT I DO, I SELECT MEGAN OVER HERE. THIS WATER FILLED CHART IS COOL. WHAT IT GIVES US, ROW LEVEL PREDICTIONS BUT EXPLANATIONS. IT IS GIVING US ROW LEVEL EXPLANATIONS, HOW IT DERIVED EACH OF THE PREDICTIONS THAT WE SEE INSIDE THE REPORT. SO WE ARE TRYING AGAIN TO BRING AS MUCH TRANSPARENCY TO THE MODELS AS WE CAN. FOR EXAMPLE, HERE, WE CAN SEE THAT MEGAN IS USING THE MOBILE APP FOR INTERACTING THE CLOUD SERVICE ACTUALLY INCREASING THE RISK THE FACT THAT SHE OPEN ADD LOT OF TICKET, YOU CAN SEE THAT I HOVER OVER THIS, THE TICKET COUNT IS HIGH. ALSO INCREASING HER RISK. THE THINGS ON THE RIGHT HAND SIDE, MAYBE THE FACT THAT SHE IS IN A SMALLER COMPANY, REDUCING HER RISK. THAT IS HOW WE ESSENTIALLY DERIVE THIS NUMBER OF THIS. WE CAN SEE PUSHING IT UP AND DOWN. FIRST, IT GIVES US TRANSPARENCY INTO THE MODEL. SECONDLY, IT ALSO GIVES US MAYBE SOME IDEAS OF HOW WE CAN INSURE THAT MAYBE MEGAN HAS A BETTER EXPERIENCE IN THE FUTURE. SO AGAIN, IF YOU WANT TO LEARN MORE ABOUT THE TECHNOLOGIES , ADVERTISE MY SESSION, A LOT MORE DETAIL OF POWER BI AND POWER AI. WITH THAT, I WANT TO JUMP BACK INTO THE SLIDES. JUST DO A LITTLE BIT OF A RECAP OF, OH, ACTUALLY BEFORE THE RECAP, A LOT OF, I GET THIS QUESTION A LOT. I DID ADD THIS SLIDE IN. OF WHAT IS, WHEN DO PEOPLE USE AUTOMATED ML AND POWER BI AND WHEN IN AZURE. AS WE INTEGRATE MORE CLOSELY BETWEEN AZURE AND POWER BI, WE GET A LOT MORE OF THESE QUESTIONS. WE WANT TO PROVIDE CLARITY ON WHEN, YOU KNOW, SOMEONE WILL BE USING WHICH TECHNOLOGY BECAUSE, YOU KNOW, THE CAPABILITIESS ARE VERY, MAYBE THEY OVERLAP BUT THERE IS, YOU KNOW, CLEAR USE CASES FOR WHEN YOU WOULD USE ONE VERSUS THE OTHER. FIRSTLY, PERSONA USE CASE. THE FACT THAT AUTOMATED ML AND POWER BI IS MEANT FOR THE ANALYST VERSUS THE DATA SCIENTIST WHO WOULD BE USING IT. OR THE DATA ENGINEER WHO WOULD BE USING IT INSIDE AZURE. THE SKILL SETS LINKED TO THE PERSONA IS DIFFERENT. ONE IS, YOU KNOW, POWER QUERY, DATA FLOWS, MORE OF A KIND OF DRAG AND DROP. KIND OF TYPE OF ANALYSIS. WHERE INSIDE AZURE IS REALLY WORKING WITH THINGS LIKE AZURE NOTEBOOKS, PYTHON, SO VERY DIFFERENT SKILL SET. ANOTHER THING THAT YOU MAY WANT TO CONSIDER IS WHERE YOU WANT TO USE THIS MODEL WHEN YOU ARE THINKING ABOUT HOW YOU ARE GOING TO OPERATIONALIZE IT. IF THIS MODEL IS PURELY FOR ANALYTICS AN USING USING IT FOR JUST POWER BI THEN POWER BI IS PROBABLY WHERE YOU WANT TO CREATE YOUR MACHINE LEARNING MODEL. IF YOU WANT TO PLUG IT INTO OTHER BUSINESS PROCESSES, MAYBE THINGS LIKE, YOU KNOW, LOGIC CAPS OR SCORING, YOU KNOW, WITH OTHER DATA OUTSIDE OF POWER BI. THEN PROBABLY AZURE, IF YOU WANT TO USE IN MODEL IN OTHER PLACES. FINALLY A DIFFERENT SUBSCRIPTION MODEL FOR POWER BI. LOOKING AT POWER BI PREMIUM. IN AZURE, IT IS LOOKING AT REALLY LEVELING THE AZURE SUBSCRIPTION. WITH THAT, JUST TO WRAP UP, I WANT TO KIND OF QUICKLY SHOW YOU GUYS THE SLIDE. TALK A LITTLE BIT ABOUT ALL OF THE TECHNOLOGIES THAT WE TALKED ABOUT TODAY. WE SHOWED YOU HOW YOU CAN HAVE KIND OF PEEL THE ONION A LITTLE BIT, START WITH A HIGH LEVEL USE CASE OF THE BUSINESS USER INTERACTING WITH SOME OF THEIR BUSINESS APPLICATION DATA. TO THEN USING POWER BI FOR SELF-SERVICE DATA PREP. FOR DOING INTERACTIVE ANALYSIS ON TOP OF THINGS LIKE, YOU KNOW, SQL CW AS WELL AS LEVERAGING, YOU KNOW, THINGS LIKE AUTOMATED ML, INSIDE POWER BI. HERE I WANTED TO HIGHLIGHT IS POWER BI IS OFTEN THE ENTRY POINT TO START BRINGING IN DATA. ITERATING ON TOP OF OUR DATA. IT CAN ALSO BE KIND OF THE FEEDBACK LOOP BACK INTO POWER BI ARV WE HAVE DONE MORE COMPLEX TRANSFORMATIONS SUCH AS WE SAW WITH THE DATA WAREHOUSE SCENARIO OR THE AI SCENARIO WHERE WE GO BACK INTO POWER BI AND VISUALIZE THE RESULTS. AGAIN, MAYBE WE ARE BIAS BECAUSE WE ARE FROM THE POWER BI TEAM. AZURE DATA SERVICES ARE SO POWERFUL AND SO MUCH YOU CAN DO TO TRANSFORM AT PREDICTIVE POWER, I THINK THAT POWER BI PROVIDES THAT VISUALIZATION THAT OFTEN REALLY BRINGS THE MESSAGE TO THE BUSINESS USERS. WHEN THEY SEE IT IN POWER BI, THEY REALLY GET IT. VERSUS WHEN YOU SHOW THEM CODE OR EVEN, YOU KNOW, I WAS AN R USER BEFORE I WAS A POWER BI USER, I WAS LIKE, I DON’T NEED POWER BI, I CAN DO EVERYTHING IN R. BUT REALLY, BEING ABLE TO PROVIDE THAT REALLY NICE INTERFACE, INTERACTIVE ANALYTICS AN BEING ABLE TO PLUG IN PREDICTIVE THINGS INTO POWER BI, REALLY PROVIDES THAT BUY-IN FROM THE BUSINESS IT IS SUPER IMPORTANT. AZURE DATA FACTORY AND HOW THAT COMPLEMENTS POWER BI FOR THAT DATA INGESTION AND TRANSFORMATION LAYER. LEVERAGING THINGS LIKE DATA BRICKS AND AZURE MACHINE LEARN FORGE DATA PREPARATION AND MACHINE LEARNING AND THEN YOU KNOW STORING ALL OF CURATED DATA AND AZURE CW. REALLY IMPORTANT THING IS AZURE DATA LINK STORAGE AT THE BOTTOM IS KIND OF THE GLUE FOR THE COMMON DATA MODEL WHERE ALL OF THAT DATA IS BASICALLY BEING STORED. AND ALL OF THESE DATA SERVICES ARE INTERACTING ON TOP OF THAT DATA. SO SESSION TAKE AWAYS, YOU KNOW, INTEGRATING WITH OTHER DATA SERVICES ON TOP OF THAT UNIFIED DATA LINK FOR THAT CROSS COLLABORATION. CHRISTIAN SHOWED YOU THAT INTERACTIVE ANALYSIS OVER PETABYTE OVER DATASETS. LEARN MORE ABOUT THAT, CHECK OUT CHRISTIAN’S CENTER. WHETHER IN AZURE OR POWER BI, WE’LL COVER MORE OF THAT TOMORROW AS WELL. SO HERE ARE THE SESSIONS IF YOU WANT TO CHECK OUT ANY OF THE OTHER ONES WE HAVE. WE HAVE FIVE MINUTES LEFT. IF YOU GUYS HAVE ANY QUESTIONS WE WOULD LOVE TO TAKE THEM NOW. THANK YOU . WE HAVE MICS ON EITHER SIDE. IT WOULD BE EASIER SO EVERYONE COULD HEAR THE QUESTION. IF YOU DON’T MIND POPPING OVER. THANK YOU.>> THE PIPELINES YOU SHOW, ALSO AVAILABLE IN EMBEDDED? BECAUSE IN SUITE, WE FACE THAT PROBLEM. WE HAVE, A TON OF DATA BUT OUR USERS WANT TO BRING THEIR DATA IN. WANT TO MASH UP. CREATE THEIR OWN DASHBOARDS, IS THAT AVAILABLE? COMING?>>YES, SO DATA FLOWS, WE HAVE A LOT OF ISBs WHO ARE EXTENDING THE COMMON DATA MODEL WITH THEIR OWN ENTITIES. AND, YOU KNOW, BUILDING APPLICATIONS, LEVERAGING DATA FLOWS, AND THEN CREATING BASICALLY EMBEDDED APPLICATION FORCE THEIR USERS. I’M JUST TRYING TO THINK IF THERE ARE ANY SESSIONS THAT COVER THAT. WE DID A SESSION YESTERDAY SO YOU COULD CHECK IT OUT, MAYBE ON DEMAND AROUND HOW KIND OF THE MORE FROM THE POWER PLATFORM AND ISV SCENARIO AND HOW THE COMMON DATA MODEL PLAYS IN WITH DATA FLOWS AND THAT. BUT I’M NOT SURE IF THERE ARE ANY OTHER SESSIONS WHERE WE GO INTO, THERE IS A POWER BI SESSION THAT, YOU KNOW, MIGHT BE WORTH CHECKING OUT. I’M NOT SURE IF COVERING DATA FLOWS AN CDM THERE. THEY WILL BE THE RIGHT PM TO TALK TO IN MORE DETAIL ABOUT THAT ASKING WELL TOO.>>DATA FLOW PREMIUM ONLY? AVAILABLE?>>THE QUESTION IS, IS DATA FLOW PREMIUM OR ALSO AVAILABLE IN PRO? DATA FLOW IS AVAILABLE IN PRO. SOME OF THE FEATURES OF THE DATA FLOWS ARE ONLY AVAILABLE IN PREMIUM THOUGH. THE ABILITY TO LINK DIFFERENT ENTITIES TOGETHER IS A PREMIUM FEATURE BECAUSE THAT REQUIRES ADDITIONAL COMPUTATION. TO CREATE DATA FLOWS TO HOOK IT UP TO AZURE DATA LINK, THAT IS AVAILABLE IN PRO. YEAH, THE COMMON DATA, STORING THE DATA AND COMMON DATA MODELS AVAILABLE IN POWER DATA FLOW. YES. >>IF WE NEED TO DO SOME CUSTOM TRANSFORMATION ON THE DATA BEFORE, EITHER BEFORE IT GOES UP, WHAT POINT WOULD YOU DO CUSTOM? SO THE DATA IS IN SQL SERVER, FOR EXAMPLE, BUT SOME PROPRIETY DATA IN THERE THAT NEEDS TO BE TRANSFORMED. WHERE WOULD YOU DO WITH THAT?>>AGAIN, DEPENDING ON WHICH PERSONA YOU WERE, WHETHER YOU WANTED TO USE POWER BI WITH DATA FLOWS OR ADF, AZURE DATA FACTORY, DATA ENGINEER, THAT IS REALLY WHERE, YOU KNOW, YOU WOULD BE ESSENTIALLY PREPARING YOUR DATA BEFORE IT LANDS INSIDE THE REPORTS. SO I DIDN’T REALLY GO INTO A LOT OF THE DATA TRANSFORMATION CAPABILITIES OF DATA FLOW THAT SHOWED HOW YOU CAN INGEST THE DATA AND STORE IT. BUT POWER QUERY HAS OVER 300, I WANTED TO SAY TRANSFORMATIONS, YOU KNOW, THAT YOU ARE ABLE TO CARRY OUT ON TOP OF THE DATA. WHETHER IT IS REMOVING COLUMNS OR CREATING NEW CUSTOM COLUMNS BASED ON THE DATA. OR, YOU KNOW, REPLACING VALUE, REMOVING NULLS, THOSE SORT OF THINGS.>>CUSTOM TRANSFORMATION. PROPRIETY, THERE IS A FIELD AND PROPRIETY FORMAT THAT YOU WANT TO EXTRACT, YOU KNOW, WE HAVE SOME C SHARP CODE THAT WILL DO THAT EXTRACTION, FOR EXAMPLE. AND YOU CAN RUN THAT ANYWHERE?>> YOU WOULD LOOK TO RUN THAT INSIDE AZURE DATA FACTORY.>>YEAH, THERE IS A . NET COMPONENT FOR AZURE DATA FACTORY. YOU CAN LOAD THE DATA INTO STAGING DATABASE. AND YOU CAN, YOU KNOW, RUN ELT PROCESSES ON THE DATA SO, YES, YOU CAN WRITE IT BACK IN CDM FORMAT TO BE CONSUMED IN DATA PROCESS. THANK YOU.>>THANK YOU. >>OKAY. GO AHEAD.>>ONE MORE? >>WELL, THE DATA FACTORY AND THE DATA FLOWS IN YOU BOUNCE BETWEEN THEM. SO CAN I START WITH THE DATA FLOW? AND THEN ACCESS THE UNDERLYING ADF AND DEVELOP AT FURTHER, USE DATA FLOW REALLY IS FACILITATING A STARTING POINT?>>NOT, THERE IS NO WAY OF KIND OF MIGRATING THE, FROM ONE TO THE OTHER. BUT THEY BOTH WOULD STORE THE DATA IN THE CDM FORMAT. SO THEY WOULD BE BOTH ABLE TO CONSUME THE OUTPUT FROM EACH OTHER. NO WAY OF MIGRATING THE PACKAGE FROM ONE TO THE OTHER. >>OUR RECOMMENDATION WOULD BE, START WITH ONE, WRITE THE DATA INTO CDM. THEN THE OTHER CAN JUST HAVE THAT AS A NEW DATA FLOW AND CREATE ENRICHED DATA.>>YEP.>>RIGHT. >>HELLO.>>YOU MENTIONED EXTERNAL DATA FLOWS. CAN YOU EXPAND ON THAT? >>YEAH, I CAN QUICKLY SHOW THAT AND SHOW WHAT KIND OF THAT LOOKS LIKE. SO LET ME JUMP INTO POWER BI FOR A SECOND. I’M NOT SURE IF IT HAS BEEN, SO, HERE BASICALLY, IF I FIND NEW ENTITIES, I CAN ATTACH A COMMON DATA FOLDER WHICH IS IN PREVIEW. THIS CAN BASICALLY, I CAN ESSENTIALLY CREATE A NEW DATA FLOW BASED ON SOMETHING THAT IS KIND OF EXTERNAL IF I HAVE THE FOLDER PATH BASICALLY PROVIDED OVER HERE. SO THIS IS BASICALLY, YEAH, FEATURING PREVIEW WHERE THIS WOULD BE READING IN AN EXTERNAL, AN EXTERNAL DATA FLOW FROM ANOTHER, ANOTHER SOURCE SUCH AS ADF IF I WANTED TO DO THAT TOO.>>OKAY. ALL RIGHT. WELL THANK YOU VERY MUCH EVERYONE. ENJOY THE REST OF THE CONFERENCE.>>THANK

Leave a Reply

Your email address will not be published. Required fields are marked *