Oracle to Azure PostgreSQL database migration webinar

Editor's Notes

• #2: Dms 확인
• #4: 감사 환영, 감사 자기소개 DMIAD 워크샵 2회, 지난 웨비나에서는 SQL Server Azure SQL DB Migration 오늘의 주제 대상: 현재 오라클, Postgre -> 클라우드 이관 오늘 웨비나에서 다룰 내용 오라클/PostgreSQL 사용자 클라우드 마이그레이션 옵션들 (간략) (오늘의 메인 주제) PaaS버전 Azure PostgreSQL 소개와, Azure DMS (Database Migration Service) 활용하면 Migration을 얼마나 간편하게 할 수 있는지 중간중간 화면 전환하여 직접 데모와 함께 보여드리겠습니다.
• #5: 요즘 PostgreSQL 많이 사용하고 계시죠. 선정 풍부한 기능으로 엔터프라이즈에서도 사용 특히 오라클 사용자 중 비용 절감/ 오픈소스 도입의 과제 -> 가장 많이 선택 오라클과 비슷한 부분이 多 -> DB 이관, 마이그레이션 하실 때 가장 적은 공수 PostgreSQL has gained credibility of enterprise ready and feature-rich database Reduce total cost of ownership (TCO) Shifting to adopt open source Similarities between Oracle and PostgreSQL to ease effort of migration Sources: https://insights.stackoverflow.com/survey/2019?utm_source=so-owned&utm_medium=blog&utm_campaign=dev-survey-2019&utm_content=launch-blog https://db-engines.com/en/blog_post/76 https://db-engines.com/en/ranking_trend/system/PostgreSQL
• #6: 클라우드에서 postgre SQL 사용 시 관리, 보안, 성능 확장, 그리고 글로벌서비스 측면에서의 장점을 누릴 수 있음 Microsoft 클라우드 Azure로 이관 시 선택 옵션 Oracle 사용자 -> Azure VM 위 Oracle 직접 설치 Oracle/기존 OSS Postgre 사용자 -> Azure VM 위 오픈 소스 PostgreSQL (IaaS) Azure Database for PostgreSQL (PaaS)
• #8: 참고
• #9: Postgre -> VM 위 IaaS IaaS와 비교하여 PaaS 서비스이기 때문에 가지는 장점 주요 장점 고가용성, 성능, 확장 등 Microsoft 완전하게 관리. 스토리지/컴퓨팅 성능을 따로 자유로이 Intelligent 성능 최적화 3. 마이크로소프트 개발자들이 PostgreSQL 커뮤니티버전에 기여한 확장 기능들 포함하여 오픈소스 엔진 그대로, 가장 최신 버전까지 사용 가능. 리소스 생성 시 배포 옵션 2가지 PostgreSQL 개발자들이 차린 회사 Citus를 인수 샤딩을 지원하는 고성능 서버그룹 -> Hyperscale 사용할 수 有 Overview: Microsoft has a numerous database services from open source to SQL; all with built in intelligence, flexibility and trust you expect from an Azure PaaS offering. Talking Points: We’re uniquely positioned to address the complexity our customers face because we see ourselves as a data platform company, not an engine company Our relational cloud assets are all built on the same platform Our aspiration is that platform innovations are shared across engines, so customers can leverage the features that make them more productive in the engine of their choice. Our strategy is built upon pillars that uniquely differentiate us in the market. We provide scalable, performant, secure and intelligent relational databases for: Born in the cloud applications and Existing applications which are either being modernized on-premises or moving to the cloud. Let’s walk through the pillars: Hybrid – we’re providing a frictionless migration experience for existing apps, whether moving to a fully-managed database as a service or transitioning over time with a hybrid strategy. Enterprise Scale and Performance – we’re helping customers manage their resources and build for the future with dynamic scaling up to 100TB. Security & Compliance – Security management can be complex, particularly when working across entire data estates. We are simplifying security with a consistent and comprehensive policy-based approach across the platform Built-in Intelligence – we’ve been enabling customers to be more productive and gather new insights with adaptive and ML-based features for a couple years now. We gather telemetry across millions of databases to fine tune our algorithms to do more and help our customers be more productive than ever. Choice - Our platform is under-girded by choice that guides customers to the right solution for their workloads at the best TCO. Customers can exercise choice and flexibility across the relational database platform, and be assured that they can maximize productivity, efficiency and ROI for any of their workloads. -------------------------------- Choice of hosting – on-premises, hybrid, VM or fully-managed PaaS Choice of engine – SQL, PostgreSQL, MySQL, Maria DB Choice of deployment options – instance and database scoped, compute and IO-intensive Choice of resources – wide spectrum of compute and storage Choice of languages - Python, Java, Node.js, PHP, Ruby, and .NET
• #10: 지금부터는 1. 기능. 고가용성 및 비즈니스 지속성 성능 및 확장성 보안 및 관리 설명 2. 마이그레이션 방법 보여드리고 3. 사례 소개
• #11: Azure Database for PostgreSQL의 고가용성 및 비즈니스 지속성
• #12: 99.99% SLA 지원 Azure SQL DB와 마찬가지, Service Fabric 기반 기술 사용 복잡한 failover 구성, 비싼 고가용성 솔루션 없이 Azure Database Services is built upon the SQL Database platform which is a Service Fabric-based PaaS solution. As such, rather than having to boot-up an entire OS stack to bring up a new server (such as in IaaS), Azure Database Services run the database engine in a custom container technology which you can think of as a secure “pico process”. The time it takes to bring-up a new server in this custom container is a matter of seconds. This means that in the event that your database server has hung, or “gone away”, the Azure Database Management Service” detects the failure, brings-up a new server in this lightweight container, maps the new IP address to the DNS name of your instance and maps to your storage. This entire process takes between 30-45 seconds. This is built-in to all performance tiers of Azure Database Services and since a replica instance isn’t needed, there is no additional cost to the customers. In contrast, an AWS RDS server that is deployed in a single AZ would take minutes to start – and that does not account for how you would detect the failure and switch-over Scale compute up or down in seconds Scale storage instantaneously High availability without the need for replicas Setting-up high availability for database servers is hard, requiring either custom code to manage detection/failover, or expensive 3rd party solutions to make it a bit easier. Compute redundancy: If a node-level interruption occurs, the database server automatically creates a new node and attaches data storage to the new node. Any active connections are dropped and any inflight transactions are not committed. Data reliability: 3 copies of data for data reliability
• #13: 오라클 -> 최대 95% 비용 절감 가능. 같은 Azure 내의 IaaS와 비교해서도 비용측면에서 장점 누릴 수 있음. 타 클라우드 벤더사 PaaS DB -> VM 형식 Customers who are IaaS customers in Azure today to understand that the specs of a VM do not equate directly to the specs of Azure Database Services. The reason is two-fold: Customers do not size a VM based on their typical workload, rather they size wisely to handle workload spikes so as not to impact performance of the application. With Azure Database Services, the ability to scale performance on the fly means they SHOULD size their instance based on typical workload needs and then elastically scale when necessary. This lowers costs. A VM has to support the performance requirements for both the database engine as well as the host OS. With Azure Database Services, the SQLPAL isolated pico-process (mini-OS) significantly lowers the HW needs compared to a VM. So in this example, if I have a D4S_V3 VM with 4 vCPUs and 32GB of SSD, when I choose an Azure Database for MySQL the customer can likely choose a smaller size of 2 vCores with the same storage (and in fact, they would get more storage as the storage for Azure Database Services is dedicated to the database, logs, etc. – no host OS footprint here). The customer can then profile their workload and determine if it meets their performance requirements, and if it does not, they can easily scale-up to the next tier. More importantly, in an IaaS VM implementation, if you want to achieve HA you need a second server (replica). This will double their costs, in this case from $143/mo. to $286/mo. With Azure Database Services with built-in HA, there are no additional replicas needed and as such – there is no cost impact. So to sum up this example, a HA IaaS MySQL VM costs $286/mo., whereas Azure Database for MySQL would cost $132/mo. That’s a saving of $154/mo.
• #14: Customers who are IaaS customers in Azure today to understand that the specs of a VM do not equate directly to the specs of Azure Database Services. The reason is two-fold: Customers do not size a VM based on their typical workload, rather they size wisely to handle workload spikes so as not to impact performance of the application. With Azure Database Services, the ability to scale performance on the fly means they SHOULD size their instance based on typical workload needs and then elastically scale when necessary. This lowers costs. A VM has to support the performance requirements for both the database engine as well as the host OS. With Azure Database Services, the SQLPAL isolated pico-process (mini-OS) significantly lowers the HW needs compared to a VM. So in this example, if I have a D4S_V3 VM with 4 vCPUs and 32GB of SSD, when I choose an Azure Database for MySQL the customer can likely choose a smaller size of 2 vCores with the same storage (and in fact, they would get more storage as the storage for Azure Database Services is dedicated to the database, logs, etc. – no host OS footprint here). The customer can then profile their workload and determine if it meets their performance requirements, and if it does not, they can easily scale-up to the next tier. More importantly, in an IaaS VM implementation, if you want to achieve HA you need a second server (replica). This will double their costs, in this case from $143/mo. to $286/mo. With Azure Database Services with built-in HA, there are no additional replicas needed and as such – there is no cost impact. So to sum up this example, a HA IaaS MySQL VM costs $286/mo., whereas Azure Database for MySQL would cost $132/mo. That’s a saving of $154/mo.
• #15: 로컬 중복 스토리지/ 지역 중복 스토리지 지역 복원 특정시점 복원 Reference: https://docs.microsoft.com/en-us/azure/postgresql/concepts-business-continuity https://docs.microsoft.com/en-us/azure/postgresql/concepts-backup All backups are encrypted using AES 256-bit encryption.
• #16: 성능 및 확장성
• #17: Read replicas help improve performance and scale of read-intensive workloads such as BI and analytics Consider the read replica features in scenarios when delays in synching data between the master and replicas are acceptable Create a replica in a different Azure region from the master for a disaster recovery plan, where a replica replaces the master in cases of regional disasters Data storage on replica servers grows automatically without impacting workloads Reference: https://docs.microsoft.com/en-us/azure/postgresql/concepts-read-replicas https://docs.microsoft.com/en-us/azure/postgresql/howto-read-replicas-portal
• #18: 단일서버 / 울트라 고성능 서버 그룹 배포 Single Server의 경우, 티어에 따라 차이는 있지만 최대 64vCore, vCore의 최대 메모리는 10GB. 스토리지 최대 크기 16TB, 최대 IOPS 20000까지 지원. Azure premium storage (GP, MO)
• #19: Hyperscale의 구조 코디네이터 노드, 워커노드 작업자 노드 구성으로 샤딩 지원. 최대 20개의 워커노드까지 수평적 확장 scale up. 쿼리를 병렬처리함으로써 더 빠르게 응답 Hyperscale의 경우에는 티어가 따로 없고 최대 64vCore, vCore 당 메모리는 8기비바이트. SSD 스토리지를 사용하고, 스토리지 최대 크기는 2티비바이트 최대 2티비바이트+20개 작업자 노드 = 스토리지 사이즈는 40티비바이트, IOPS는 12만2960. Aggregating data before transactions avoids rewriting each row and can save write overhead and table bloat Bulk aggregation avoids concurrency issues
• #20: 관련 테이블의 관련된 행은 동일한 노드에 위치시켜 관련 행 join 시 데이터가 불필요하게 네트워크 이동을 것을 방지.
• #21: 노드를 확장시켰을 때 과거 작업자 노드와 새 작업자 노드가 균형을 이룰 수 있도록 재분산. 트랜잭션 확장, 샤드 밸런싱 뿐만 아니라 Azure Database for PostgreSQL Hyperscale -> 파티셔닝, 병렬 인덱스, savepoint, window function 등 더 다양한 기능 지원 Transactional support Savepoint support Multi-value inserts PostgreSQL10, PostgreSQL11 Window functions Online shard rebalancing Scaled-out transactions Distinct on/count distinct CTE support Native PostgreSQL partitioning Enhanced SQL support TopN Citus MX (beta) Rename of scale-out tables Parallel index Parallel vacuum Scaled-out backups Hyperscale (Citus) Cloud Shard Rebalancer Shard rebalancer redistributes shards across old and new worker nodes for balanced data scale-out Shard rebalancer will recommend rebalance when shards can be placed more evenly For more control, use tenant isolation to easily allocate dedicated to specific tenants with greater needs
• #22: https://docs.microsoft.com/ko-kr/azure/postgresql/concepts-pricing-tiers https://docs.microsoft.com/ko-kr/azure/postgresql/concepts-hyperscale-configuration-options Single Server 티어에 따라 다르지만 최대 64vCore, vCore의 최대 메모리는 10GB Azure premium storage 사용할 수 있고, 스토리지 최대 크기 16TB 최대 20000IOPS Hyperscale 티어가 따로 없고 코디네이터 노드/ 작업자 worker 노드 (최대 20개) 최대 64vCore, vCore 당 메모리는 8기비바이트 SSD 스토리지를 사용하고, 스토리지 최대 크기는 2티비바이트 2티비바이트+20개 작업자 노드 = IOPS는 12000이상
• #23: 제가 설명드렸던 Azure Database for PostgreSQL 싱글서버/hyperscale을 생성하는 데모 PC 화면을 전환
• #25: 장표로 돌아와 기능소개를 계속하겠습니다. 보안 및 관리
• #26: 보안 기능 여러 단계 지원 기존 인증 기능, AAD -> 추가 구성없이 접근 제어 SSL 연결, 방화벽, Vnet 당연 지원 스토리지 암호화/ 네트워크를 보호. AWS Directory Services integration requires additional coding AWS Identity & Access Management requires creation of additional users GuardDuty integration requires additional configuration
• #27: 추가 선택 옵션: 고급 위협 보호 감지 -> 관리자 알림 Azure PostgreSQL threat detection provides an additional layer of security intelligence which detects suspicious activities going on in the database. A simple way to enable threat detection using Azure portal, which requires no modifications to existing application code or client applications. A proprietary set of algorithms that work around the clock to learn, profile and detect suspicious databases activities, indicating a potentially harmful attempts to access or exploit data in the database. Someone has logged from an unusual location - change in the access pattern from an unusual geographical location An unfamiliar principal successfully logged- - change in the access pattern using an unusual SQL user. Someone is attempting to brute force SQL credentials abnormally high number of failed logins with different credentials. Someone has logged from a potentially harmful application   It provides actionable alerts over email and in Azure portal which provides details of the suspicious activity and recommends how to further investigate and mitigate the threat. ---------------------------------------------------------- We are embedding machine learning directly into our cloud services to deliver intelligent data services that keep your data safe. For example, consider the security features in Azure SQL DB  Our ML systems analyze and learn from over 700 TB data/per day to ensure we keep your applications highly efficient and data safe – through automatic auditing and threat detection. With active Threat Detection, the service can identify anomalies in your workload and alert you of a potential attack like SQL injection. The service does the hard work so you don’t have to – so you can focus on the business problems you’re solving and creating breakthrough applications. --------------------------------------------------------------------------------------------------- SQL Threat Detection allows you to detect suspicious activities indicating a possible malicious intent to access, breach or exploit data in the database. SQL Database Threat Detection runs multiple sets of algorithms which detect potential vulnerabilities and SQL injection attacks, as well as anomalous database access patterns (such as access from an unusual location or by an unfamiliar principal). Security officers or other designated administrators get email notification once a threat is detected on the database. Each notification provides details of the suspicious activity and recommends how to further investigate and mitigate the threat. “Azure SQL Database Threat Detection is now generally availableThreat Detection leverages machine learning to provide an additional layer of security built into the SQL Database service, enabling SQL Database customers to protect their databases within minutes without needing to be an expert in database security. It works around the clock to profile and alert you of anomalous activities on your databases. Threat detection alerts can be viewed from Azure Security Center and provide details of suspicious activity and recommend action on how to investigate and mitigate the threat. To learn more about Threat Detection, including pricing, visit the Azure blog.    
• #28: 보안 관련 여러 certificate 보유 기업의 규정을 준수하셔야 하는 분들은 참고 Protecting your innovation in the cloud: Reduce risk, innovate with confidence, and operate with freedom in the cloud. Azure IP Advantage provides the industry’s most comprehensive protection against intellectual property (IP) risks. -Best-in-industry intellectual property protection -Build confidently with uncapped indemnification -Deter and defend lawsuits with patent pick -Get broad protection with a springing license Based on customer demand from various industry verticals SOC2 - Service Organization Controls standards for operational security ISO 27001 - Information Security Management Standards ISO 27018 - Code of Practice for Protecting Personal Data in the Cloud CSA STAR -  Cloud Security Alliance: Security, Trust & Assurance Registry (STAR) PCI DSS Level 1 - Payment Card Industry (PCI) Data Security Standard (DSS) Level 1 Service Provider HIPAA / HITECH Act - Health Insurance Portability and Accountability Act / Health Information Technology for Economic and Clinical Health Act ISO 27017:2015 - Code of Practice for Information Security Controls ISO 9001:2015 Quality Management Systems Standards ISO 22301:2012 Business Continuity Management Standard ISO/IEC 20000-1:2011 Information Technology Service Management
• #29: 모니터링 기능 내장 -> default로 모니터링 화면 사용 사용자에게 경고/ 자동알림 구성
• #30: 기본적으로 내장된 server log 기능. 간편하게 구성
• #31: 기존 PostgreSQL DB에서 복잡하게 관리하셨던 로그들은 Azure Portal에서 간편하게 관리할 수 有
• #32: 매우 유용한 성능 최적화 기능 기본 내장. 다음 3가지 Query Store 가장 오래 도는 쿼리, 가장 많은 리소스를 사용하는 쿼리 등을 바로 찾을 수 있고, Query Performance Insight 라는 기본 제공 모니터링 화면을 통해 바로 확인 가능. 인덱스 생성/삭제과 같이 성능 개선 추천사항 제공하여 여러분들의 업무시간 효율적으로 활용할 수 있도록 도움.
• #33: 클라우드의 많은 부분 활용 For application developers using PostgreSQL, Azure provides integration with popular frameworks like Drupal, Django, etc. And also popular languages like python, etc. We have done work make it simple for application developers to provision both applications and PostgreSQL with build in connection it Azure App Services and other services within Azure. We have several customers building interesting solution (which I will talk about later) building interesting scenarios leveraging advanced analytics and AI scenarios. PG has deep integration with intelligent Azure services like Cortana APIs. Our customers are building solutions to reach their customer base world wide. PostgreSQL is and will take advantage of Azure’s global reach of 50+ regions. Also we have several customers wanting to migrate off of on premises/private clouds to Azure. The Azure Database Migration service provides online migration capabilities to Azure PostgreSQL w/o the application taking any downtime. Span with Azure’s availability in more regions worldwide than any other cloud provider PBI Azure Functions
• #34: 이런 기능들을 어디서 확인할 수 있는지 보여드리기 위해 생성된 Azure DB for Postgre 함께 확인 쿼리 툴로 연결 demo 화면 전환. http://127.0.0.1:52934/browser/ SELECT version(); SELECT * FROM pg_tables;
• #40: AS-IS 소스 DB 진단/분석 타겟 DB에 맞게 전환 마이그레이션 최적화
• #41: 오프라인
• #42: 서비스 중지 시간 없이, 다운타임 없이 온라인 마이그레이션 Azure의 DB로 데이터를 이관하는 작업을 담당 PaaS
• #45: 마지막 데모 Oracle -> azure PostgreSQL 온라인 마이그레이션 준비한 DB 소스 환경은 Windows 가상머신 위 설치된 오라클 express버전 리눅스를 사용하시는 분들도 환경설정만 해주신다면 똑같이 화면 전환.
• #48: 데이터베이스 마이그레이션 전략 수립을 위한 사전진단 분석 컨설팅이 필요하신 경우 Microsoft는 데이터 전문 파트너사와 함께 하실 수 有 AS-IS 데이터베이스 진단 및 분석 스키마, 어플리케이션 전환 및 데이터 마이그레이션 검증 테스트-> 프로덕션 까지의 DB 마이그레이션 작업들을 데이터솔루션, 메타넷T플랫폼 MTP와 함께할 수 있음
• #49: 마지막으로 사례 부분입니다.
• #50: 본사 윈도우 팀 1.5PB 이상의 데이터를 저장해서 실시간 분석 활용 95%의 쿼리응답속도가 4초 미만 75%의 쿼리응답속도가 200ms 미만이라고 공개.
• #51: 관련 자세한 내용은 곧이어 영상으로 직접 보여드리도록 하겠습니다.
• #52: 제가 오늘 준비한 내용은 여기까지입니다. 현재 오라클/PostgreSQL 사용자 Azure PostgreSQL 사용하게 되면 어떤 장점들을 어떻게 누릴 수 있는지 소개 Azure DMS (Database Migration Service) 활용하여 online Migration을 얼마나 손쉽게 할 수 있는지 직접 보여드렸습니다.
• #53: 비용정보를 포함한 모든 기술문서는 오픈되어 있으니 다음 링크에서 참고부탁드립니다.
• #54: 이만 웨비나를 마치도록 하겠습니다. 경청해주셔서 대단히 감사합니다. https://www.youtube.com/watch?v=TBZdOMv8a6Q OCI Enterprise는 파티션을 지원하지 않는다 (Azure PostgreSQL은 11.5 이후부터는 완벽한 파티션을 지원) AWS의 경우 Zone Redundent를 위해서는 2개 이상의 VM이 필요하다 (> x 2 cost) AWS의 경우 SLA가 99.95% (Azure 99.99%) DMS가 online mig를 지원하기 때문에(오라클 10,11,12 지원) 최소다운타임 마이그레이션이 가능
• #57: PaaS 서비스이기 때문에 가지는 장점