Question

我正在尝试通过设置服务器 - 服务器身份验证来将本地文件加载到bigquery。我已经完成了以下步骤

创建服务帐户
为此帐户创建JSON密钥文件
带有
的已激活服务帐户
gcloud auth activate-service-account命令
使用
登录
gcloud auth login

尝试执行python脚本将文件上传到BigQuery

范围=

['https://www.googleapis.com/auth/bigquery',
         'https://www.googleapis.com/auth/bigquery.insertdata']

credentials = ServiceAccountCredentials.from_json_keyfile_name(
    '/path/privatekey.json', scopes)
# Construct the service object for interacting with the BigQuery API.
service = build('bigquery', 'v2', credentials=credentials)

# Load configuration with the destination specified.
load_config = {
    'destinationTable': {
        'projectId': "project id",
        'datasetId': "data set id",
        'tableId': "table name"
    }
}

# Setup the job here.
# load[property] = value
load_config['schema'] = {
    'fields': [
        <several field>
    ]
}


upload = MediaFileUpload('/path/to/csv/file',
                         mimetype='application/octet-stream',
                         # This enables resumable uploads.
                         resumable=True)
# End of job configuration.

run_load.start_and_wait(service.jobs(),
                        "my project id",
                        load_config,
                        media_body=upload)

结果是

   "error": {
    "errors": [
   {
    "domain": "global",
    "reason": "required",
    "message": "Login Required",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Login Required"
 }
}

但我有足够的权限创建查询作业

query_request = service.jobs()
query_data = {
    'query': (
        'SELECT COUNT(*) FROM [dmrebg.testDay];')
}

query_response = query_request.query(
    projectId=project_id,
    body=query_data).execute()

print('Query Results:')
for row in query_response['rows']:
    print('\t'.join(field['v'] for field in row['f']))

我错过了什么？我以为我已经登录了。

Answer 1

问题是任何来电 https://www.googleapis.com/bigquery/v2/projects/project_id/jobs/ *会导致同样的问题

{
  "error": {
    "errors": [
   {
    "domain": "global",
    "reason": "required",
    "message": "Login Required",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Login Required"
 }
}

所以我的浏览器auth是个问题，python auth很好。

根本原因是我的CSV架构和数据不匹配。

Errors:
Too many errors encountered. (error code: invalid)

在使用BigQuery API的python插入作业期间登录失败

1 个答案: