DQC

Connection to files on AWS S3

To connect files stored in Amazon S3 to the DQC Platform, you need to configure IAM access and supply your bucket details. This guide explains the steps and required parameters.

Add an AWS S3 connector

To get started:

  1. Navigate to the Data Connectors screen

  2. Click on the AWS S3 button

  3. Fill in the required details (see table below)

Choose AWS S3 to link your object storage bucket

Required connection information

Field

Example

Description

Name

DataLakeObjectStore

Internal name for your connection

Access Key

AKIAIOSFODNN7EXAMPLE

AWS IAM access key

Secret Access Key

{secret here}

Corresponding secret key for authentication

Region

eu-central-1

AWS region where your S3 bucket is located

S3 URL

s3://bucketname/path

Path to your data — include folder if not using root

Configure IAM access in AWS

To ensure secure and scoped access to your S3 bucket, follow these steps:

Step 1: Create IAM policy

Define a custom IAM policy with the following permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetObject",
        "s3:GetBucketLocation"
      ],
      "Resource": [
        "arn:aws:s3:::your-bucket-name",
        "arn:aws:s3:::your-bucket-name/*"
      ]
    }
  ]
}

Step 2: Create IAM user and attach policy

  1. Go to IAM > Users

  2. Create a new user or choose an existing one

  3. Attach the custom S3 access policy

  4. Generate an Access Key and Secret Access Key

Whitelist DQC Platform’s static IP

To allow inbound connections, ensure that the following static IP address is whitelisted in your S3 bucket policies or VPC security groups:

3.123.94.228

Troubleshooting

When DQC cannot connect to AWS S3 or load its tables, it shows the reason with a short fix and links to this section. Look up the beginning of the message:

Message in DQC

Likely cause

Fix

Could not connect to the data source. …

Wrong bucket or prefix, wrong access key pair, or missing IAM permissions

Check the bucket URL (s3://bucket/prefix) and the key pair; the IAM user needs s3:ListBucket on the bucket and s3:GetObject on the objects (see Configure IAM access in AWS)

Origin AWS_S3 cannot be routed through a DQC Proxy …

A DQC Proxy is selected

Remove the proxy; S3 is reached directly

Table not accessible anymore …

The file was deleted or renamed after the last sync, or its name contains *, ?, [ or ]

Restore or rename the file, then resync the connector

Request timed out

The prefix holds many or very large files; DQC reads them during the connection test

Point the connector at a smaller prefix

  • Synchronization failed: when a sync cannot connect, the connector shows Failed on the Data sources screen and the badge links to this section. Edit the connector and click Load tables or Save: DQC tests the connection again and shows the current reason.

  • Files missing from the table list: DQC lists CSV, JSON, Parquet and Excel files directly under the prefix, with lowercase extensions. Subfolders and compressed files (.gz) are not listed.

  • One broken file blocks the connector: a malformed JSON, corrupt Parquet or unreadable file makes the whole connection fail. Fix the file or move it out of the prefix.

  • Still stuck? Write to support@dqc.ai with the connector name, the time of the attempt and the full message.

💡 Notes