Skip to content

A Beginner’s Guide to AWS Redshift: What It Is and How to Get Started

Published by dbxquad on May 23, 2023

If you’ve been hearing about cloud data warehouses and wondering what all the fuss is about, you’re in the right place. Today, we’ll chat about AWS Redshift, the powerhouse Amazon uses to store and analyse massive amounts of data. Don’t worry, we’ll keep it straightforward and easy to follow—like a relaxing BBQ, but focused on data.

What Is AWS Redshift?

AWS Redshift is a cloud-based data warehouse. Think of it as a giant virtual shed where you can store all your data—only instead of keeping old bikes and garden tools, you’re storing important data that can help you make decisions. Redshift is great for quickly analysing data, running reports, and answering questions like “How did our sales perform last quarter?” or “What are our customers buying the most of?”

In more technical terms, AWS Redshift is a service that allows you to gather and organise large amounts of data and perform queries on it to gain insights. It’s perfect for businesses that want to take a deep dive into their data and come up with valuable answers. It’s also fully managed, meaning Amazon takes care of all the heavy lifting, so you don’t have to worry about things like server maintenance. Quite impressive, isn’t it?

Competitors in the Market

Of course, AWS Redshift isn’t the only data warehouse in the neighbourhood. Here are a few of the big players it competes with:

  • Google BigQuery: Google’s version of a data warehouse, designed for massive scalability and fast queries.
  • Snowflake: A popular cloud data warehouse known for its scalability and ease of use.
  • Azure Synapse Analytics: Microsoft’s offering, which integrates well with other Azure services.

Each of these solutions has its strengths, but today, we’re focusing on Redshift.

Why Use AWS Redshift?

So, why should you consider AWS Redshift? Here are some key reasons:

  • Scalable: As your data grows, Redshift can grow with you. Need more storage or more power? No worries, just add it with a few clicks.
  • Fast Performance: Redshift is optimised to process queries quickly, even with lots of data.
  • Cost-Effective: Amazon has set up Redshift to help you manage your costs. You can start small and scale as needed, paying only for what you use.
  • Integration with AWS Services: Redshift integrates seamlessly with other AWS services, like S3 for data storage and QuickSight for data visualisation.

Let’s Get Practical: Setting Up AWS Redshift Serverless

Alright, mate, enough chit-chat. Let’s roll up our sleeves and set up your first AWS Redshift namespace using the serverless option. AWS provides a special $300 credit for first-time users of Redshift Serverless, which makes it perfect for beginners who want to explore without worrying too much about costs.

Step 1: Log Into AWS

First things first, you’ll need an AWS account. Head over to aws.amazon.com and log in. If you don’t have an account, you can sign up and start using the free tier. If you’re in a corporate environment, consider using a sandbox account to explore safely.

Step 2: Navigate to Redshift

Once you’re in, type “Redshift” in the AWS search bar at the top. This will bring you to the Redshift dashboard. Click on Try Redshift Serverless free trial to start setting up your data warehouse without having to deal with complex configurations.

Step 3: Configure Your Redshift Serverless Namespace

With Redshift Serverless, you don’t need to worry about node types or cluster sizes—AWS handles all of that for you. You’ll see two options: Use default settings and Customize settings. If you select Use default settings, the namespace will be automatically named as ‘default-namespace’, and you won’t be able to modify it. For now, just choose Use default settings to keep things simple and let AWS handle the configuration.

By the way, you might be wondering what a ‘namespace’ is. Well, think of it as your very own data headquarters—a place where all the magic happens. It’s where you store, manage, and query your data, without needing to fuss over the infrastructure. It’s like having a fancy kitchen where all you need to do is cook, and AWS takes care of all the cleaning up after. How good is that?

Next, You Will See Several Other Configuration Options:

1. Database Name and Password

  • Database Name: By default, AWS Redshift Serverless creates a database named dev. You can’t change this name if you selected Use default settings. If you choose Customize settings, you may have the flexibility to modify it.
  • Admin User Credentials: An admin user is automatically created, and the password is managed through your IAM permissions.

2. Associated IAM Roles

  • Create or Associate an IAM Role: You will see an option labeled Associated IAM roles. If you don’t have an IAM role yet, click Create IAM role. This button will guide you through creating a role with the required permissions (AmazonRedshiftAllCommandsFullAccess policy). Follow the steps to create this role.
  • Attach IAM Role: Once you’ve created the IAM role, you will need to associate it with your Redshift namespace. To do this, select the newly created role from the list and set it as the default.
  • Why Is This Important?: The IAM role is like giving permissions to your Redshift namespace to communicate with other AWS services—think of it as giving your data warehouse a set of keys to access different parts of your AWS ecosystem.

Step 4: Workgroup Configuration

Alright, let’s talk about workgroups—in plain English, it’s like the engine room of your data warehouse. This is where all the computing power is organized. You’ve got a few settings here, but don’t stress, just follow these simple steps:

  • Workgroup Name: It’s set to default-workgroup. Nothing to worry about here—leave it as it is.
  • Capacity: The default capacity is 128 RPUs, which is enough to get you started. It’s basically how much computing power you’re assigning.
  • Network and Security: AWS automatically assigns a VPC (Virtual Private Cloud), subnets, and security groups for you. This keeps your data safe from any internet nasties. Again, leave these as-is unless you have a good reason not to.

Since we’re keeping things easy, no need to tweak anything. Just click Save Configuration and move on!

Step 5: Launch Your Redshift Serverless Environment

Once you’ve configured the namespace and workgroup, click Continue. Redshift Serverless will now spin up your data warehouse environment. This may take a few minutes, so grab a cuppa while you wait.

With Redshift Serverless, there’s no concept of a traditional cluster anymore—you’ll be working with your namespace and workgroup instead. Once the environment is ready, it’s time to start using it!

Step 6: Connect to Your Namespace

Once your serverless namespace is up and running, you’ll want to connect to it. AWS provides a tool called Query data (Query Editor v2) right in the Redshift console that allows you to interact with your namespace. Alternatively, you could use tools like SQL Workbench or pgAdmin if you want a bit more control.

To connect using the Query Editor:

  • Go to your Redshift namespace details.
  • Click on Query data.
  • You will see an item named Serverless: default-workgroup listed on the left. Click on it.
  • A pop-up window titled Connect to default-workgroup will appear, where the dev database is selected by default.
  • Click on Create connection to start querying the dev database.

Step 7: Create a Table and Load Some Data

Now that you’re connected to the dev database, let’s create an empty table to hold the data from our CSV file:

  • Create a Table: Use the following SQL command to create a table:
CREATE TABLE my_table (
  id VARCHAR(50),
  title VARCHAR(255),
  type VARCHAR(100),
  genres VARCHAR(255),
  averageRating DECIMAL(3, 1),
  numVotes INT,
  releaseYear INT
);

This schema is designed to align with the structure of the IMDB dataset, with appropriate data types for each column.

Now that we have a table ready, let’s load some data! We’ll be using a CSV file from Kaggle, and you’re free to create a Kaggle account or use a dataset we’ve prepared to make your life easier. You can find the dataset here: IMDB Full Dataset. Redshift Serverless can pull data directly from AWS S3, so once you have the CSV, you can easily copy it into Redshift.

dataDownload
  • Upload your CSV file to an S3 bucket.
  • Use the COPY command to load the data into Redshift. Here’s an example command:
COPY my_table
FROM 's3://my-bucket-name/my-file.csv'
IAM_ROLE 'arn:aws:iam::your-iam-role'
FORMAT AS CSV
DELIMITER ','
QUOTE '"'
IGNOREHEADER 1;

Just replace my_table, my-bucket-name, and your-iam-role with your actual details.

Step 8: Run Some Queries

With data in your table, you can now run some basic queries. Try something like:

SELECT * FROM my_table LIMIT 100;

This will show you the first ten rows of your data—kind of like a sneak peek to make sure everything looks alright. The Query Editor v2 makes it easy to run queries and see your results instantly.

Important Note: Avoid Unnecessary Costs

Once you’re done exploring, it’s a good idea to delete the namespace and workgroup to avoid incurring unnecessary costs. Redshift Serverless automatically scales resources, but you can still be charged for keeping the environment active. To delete:

  • Go to the AWS Redshift console.
  • Delete both the namespace and workgroup.

This will ensure you’re not charged for resources you’re no longer using.

Wrapping Up

And there you have it! You’ve just created your first AWS Redshift Serverless workspace, loaded some data, and run a query. AWS Redshift Serverless is a powerful tool for storing and analysing large datasets, and while we’ve just scratched the surface today, you’re now ready to start exploring on your own.

Remember, data warehousing isn’t just for big companies—it’s for anyone who wants to make sense of their data and make better decisions. And hey, who doesn’t want that?

So grab a cold one and celebrate 🎉

Published inData Warehouse