With cloud storage becoming the de facto data lake, it's almost quaint to recall that barely a few years back, SQL-on-Hadoop was a hotly-contested battleground, with well over a dozen different open source and proprietary engines duking it out. With the dust settling, as the Hadoop market winnows down with the combo of Cloudera and Hortonworks, all eyes are on how to access the growing masses of data sitting in cloud object storage. Today, each of the cloud data warehouse platforms provide ways to federate query to cloud object storage. must read Top cloud providers 2019: AWS, Microsoft Azure, Google Cloud; IBM makes hybrid move; Salesforce dominates SaaS The cloud computing race in 2019 will have a definite multi-cloud spin. Here's a look at how the cloud leaders stack up, the hybrid market, and the SaaS players that run your company. Read More But what about if you don't want to mount a Hadoop cluster or set up a data warehouse? A couple years back, AWS opened the doors with … [Read more...] about Starburst Data’s road to providing the gateway to cloud storage
Amazon s3 or google cloud storage
Apache Spark is a hugely popular execution framework for running data engineering and machine learning workloads. It powers the Databricks platform and is available in both on-premises and cloud-based Hadoop services, like Azure HDInsight, Amazon EMR and Google Cloud Dataproc. It can run on Mesos clusters too.But what of you just want to run your Spark workloads on a Kubernetres (k8s) cluster sans Mesos, and without the Hadoop YARN strings attached? While Spark first added Kubernetes-specific features in its 2.3 release, and improved them in 2.4, getting Spark to run natively on k8s, in a fully integrated fashion, can still be a challenge.Kube operatorToday, Google, which created Kubernetes in the first place, is announcing the beta release of the Kubernetes Operator for Apache Spark -- "Spark Operator" for short. Spark Operator allows Spark to run natively on k8s clusters and thus allows Spark applications -- be they purposed for analytics, data engineering or machine learning -- to … [Read more...] about Google announces Kubernetes Operator for Apache Spark
directory We found 22 cloud services your business definitely needs to try When it comes to cloud services and software-as-a-service (SaaS), we're all familiar with the usual players. But there's a huge world of opportunity and available resources beyond Google, Dropbox, Salesforce, Amazon, and Microsoft. Read More In 2007, Drew Houston, Dropbox's CEO, got sick and tired of misplacing his USB drive, so he created the first personal and small business cloud storage service. It was a radical one in its day. Today, everyone and their uncle seems to be offering cheap or free cloud storage.Also: How to sync multiple cloud storage services for free TechRepublicThat's great! Except, well, how do you choose which one is right for you? It used to be that most people decided simply on the basis of how much free storage space they got. That's simple, but it only tells part of the story.The real value from a cloud storage service comes from how well it works for you. As you'll see, some … [Read more...] about What’s the best cloud storage for you?
In the course of a single generation, Amazon has grown from fledgling online bookseller to one of the most valuable and powerful corporations in modern history. The empire of CEO Jeff Bezos has grown so vast that critics, overseas regulators, and Washington politicians are all now wondering whether the company has become an unstoppable force, and what, if anything, is capable of reigning in its reach. A recent spat with Sen. Bernie Sanders (D-VT) resulted in a minimum wage hike for tens of thousands of employees, but Amazon still operates largely without any meaningful checks on its power even as it aggressively expands into physical retail, the smart home, and warehouse and aviation robotics. Yet despite having a hand in so many different industries, consumers largely trust Amazon with everything from their personal information and buying habits to the literal conversations they have in their own homes. According to a study The Verge conducted in partnership with consulting firm … [Read more...] about How Amazon’s retail revolution is changing the way we shop
must read What is cloud computing? Everything you need to know from public and private cloud to software as a service An introduction to cloud computing from IaaS and PaaS to hybrid, public and private cloud. Read More Cloud vendors, like Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure, all offer a variety of cloud services, ranging from high-performance, SSD-based capacity, to long-latency archive storage, at prices ranging from high to relatively low. But most applications have a variety of I/O needs, from latency-sensitive metadata updates, to bandwidth sucking backups. No single cloud storage service is ideal.Application developers know this, and often perform unnatural acts in their code to overcome cloud storage deficits. Two major issues are cost/performance tradeoffs, and inelastic deployment boundaries. Cost/performance tradeoffs The storage hierarchy - in simpler times DRAM, disk, and tape - reflects the tradeoffs. Fast storage is … [Read more...] about Can cloud storage replace your onsite storage?