createTempUploads
Last updated
Last updated
This endpoint can be used for creating a collection of temporary upload endpoints, allowing for the performant and robust transfer of files to Redivis. Larger uploads can specify the resumable option, which allows for upload progress recovery in the case of intermittent network failures.
These temporary uploads can then be used both by either upload.post to upload a tabular file to a table, or file.post to add non-tabular files to a file index table.
Data uploads can be quite complex. It is highly recommended to use one of the client libraries to upload your files (and handle resumable uploads for you), rather than calling this endpoint directly.
Parameter | Type | Description |
---|---|---|
This endpoint extends the general API structure.
Edit access to the table's dataset is required. Your access token must have one of the following scopes:
data.edit
Learn more about authorization.
The request body must be empty.
Returns an object whose results
attribute is a list of temp uploads. The order of results
will always correspond to the order of the tempUploads
array in the request body.
To perform a temp upload where resumable=false
, simply PUT
the upload contents to the URL returned on the tempUpload.
First, initiate the resumable upload by submitting a POST request with an empty body to the tempUpload.url
. This post request must specify two-headers:
x-upload-content-length
: The size, in bytes, of the upload
x-goog-resumable
: "start"
If successful, the response from this initial request will include a Location
header, which specifies the URL for all subsequent requests.
Next, perform the resumable upload. The general process for performing a resumable upload is to send the file in a series of chunks, via PUT
requests to the url returned when creating the resumable upload. For each request, specify the Content-Range
header to inform which chunk is currently being uploaded, and provide the content of the chunk in the request body.
If an error occurs when uploading a chunk, you can retry the upload as specified below.
Resumable uploads expire after 24 hours.
You should provide the current chunk content in the request body.
Upon successfully transferring a chunk, the response body will be empty with a status of 200.
If an upload request is terminated before receiving a response, or if you receive a 503
or 500
Service Unavailable response, then you need to resume the interrupted upload. To resume an interrupted upload:
To request the upload status, create an empty PUT request to the resumable session URI.
Add a Content-Range header indicating that the current position in the file is unknown.
For example, set the Content-Range
to */2000000
if your total file length is 2,000,000 bytes.
If you don't know the full size of the file, set the Content-Range
to */*
.
Send the request.
Process the response.
A 200 OK or 201 Created response indicates that the upload was completed, and no further action is necessary. A 308 Resume Incomplete response indicates that you need to continue uploading the file.
If you received a 308 Resume Incomplete response, process the response's Range header, which specifies which bytes Cloud Storage has received so far. You will use this number in the next step. The response does not have a Range header if Cloud Storage has not yet received any bytes. For example, a Range header of bytes=0-42 indicates that the first 43 bytes of the file have been received.
Now that you know where to resume the upload, continue uploading the file, either by sending the remaining data or by sending the next chunk. Include a Content-Range header indicating which portion of the file you are sending. For example, Content-Range: bytes 43-1999999/2000000 indicates that you are sending bytes 43 through 1,999,999.
Header | Description |
---|---|
Parameter
tableReference
A qualified reference to the table. See referencing resources for more information.
tempUploads
Array<object>
Required. An array of objects, where each object corresponds to a single temp upload.
tempUploads[].size
integer
Required. The size of the upload, in bytes.
tempUploads[].name
string
Required. The name of the upload, unique to the current table.
tempUploads[].resumable
bool
Optional, default false. If specified, the created temp upload will be a resumable upload(see below). Resumable uploads have some minor overhead, and are generally only recommended for larger files (> 100MB)
Content-Range
Set to show which bytes in the file you are uploading.
For example, Content-Range: bytes 0-524287/2000000
shows that you are uploading the first 524,288 bytes (256 x 1024 x 2) in a 2,000,000 byte file.
If the total file size isn't known beforehand (for example, during streaming uploads that might perform processing on the stream), you can specify an unknown total size as: Content-Range: bytes 0-524287/*
If you are resuming an interrupted upload and want to know its progress, set the left part of the range to a *
to signal that it is unknown. E.g., Content-Range: bytes */2000000