要将AWS Glue连接到RDS MySQL,您可以使用以下步骤和示例代码:
步骤1:创建IAM角色
首先,您需要创建一个IAM角色,该角色具有适当的权限来访问RDS MySQL数据库。您可以使用以下代码创建IAM角色:
import boto3
iam = boto3.client('iam')
trust_policy = {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "glue.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
role_name = 'MyGlueRole'
try:
response = iam.create_role(
RoleName=role_name,
AssumeRolePolicyDocument=json.dumps(trust_policy)
)
role_arn = response['Role']['Arn']
print(f"Created IAM role: {role_name} with ARN: {role_arn}")
except iam.exceptions.EntityAlreadyExistsException:
response = iam.get_role(RoleName=role_name)
role_arn = response['Role']['Arn']
print(f"IAM role: {role_name} already exists with ARN: {role_arn}")
步骤2:为IAM角色授予适当的权限
在IAM控制台中,为刚刚创建的IAM角色添加以下策略,以便允许Glue访问RDS MySQL数据库:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"rds-db:connect"
],
"Resource": [
"arn:aws:rds-db:region:account-id:dbuser:db-instance-id/db-username"
]
}
]
}
请替换region,account-id,db-instance-id和db-username为您的RDS MySQL数据库的实际值。
步骤3:创建Glue连接
接下来,您可以使用以下代码创建Glue连接:
import boto3
glue = boto3.client('glue')
connection_name = 'MyRDSConnection'
connection_description = 'My RDS MySQL Connection'
connection_type = 'JDBC'
connection_url = 'jdbc:mysql://my-rds-instance.mydomain.com:3306/mydatabase'
username = 'myusername'
password = 'mypassword'
response = glue.create_connection(
ConnectionInput={
'Name': connection_name,
'Description': connection_description,
'ConnectionType': connection_type,
'ConnectionProperties': {
'JDBC_CONNECTION_URL': connection_url,
'USERNAME': username,
'PASSWORD': password
},
'PhysicalConnectionRequirements': {
'SubnetId': 'subnet-12345678', # Replace with your subnet ID
'SecurityGroupIdList': ['sg-12345678'] # Replace with your security group ID
}
}
)
connection_arn = response['Connection']['ConnectionArn']
print(f"Created Glue connection: {connection_name} with ARN: {connection_arn}")
请替换my-rds-instance.mydomain.com,3306,mydatabase,myusername,mypassword,subnet-12345678和sg-12345678为您的RDS MySQL数据库的实际值。
完成这些步骤后,您将成功地将AWS Glue连接到RDS MySQL数据库。您可以在Glue作业中使用此连接来读取和处理RDS MySQL数据。
上一篇:AWS Glue JSON限制