Introducing the CSV Reader - Ingest and read huge flat files faster and error-free

Greetings Citizen Automators! During 2018, our product team released our CSV Editor, which made it possible to transform and manage flat files using the Tray Platform. This feature made it possible to automate many ETL jobs like creating new CSV files, adding or deleting rows, creating transient objects, manipulating CSV files in-memory, and a myriad of other tasks within our customers’ workflows.

At Tray.io, we constantly listen to our customers’ feedback to improve the features on the Tray Platform. This is why we have recently added enterprise features like Tray for Teams, our new documentation platform, SSO for the Enterprise, and Rollback History.

After releasing the CSV Editor, our customers were thrilled with the feature and we have seen it rapidly expand in usage - in fact, it’s one of our top 50 most-used connectors within the Tray Platform! The CSV Editor’s popularity helps highlight how many of our customers need help automating these types of processes to be more successful in their roles.

While customers were extremely pleased with the CSV Editor, we realized that there were a few cases where this feature was not a perfect solution for every situation our customers face. When customers needed to simply read or extract data from extremely large flat files with, say, 1,000 columns or split up the flat file to import data to services with data limits, the CSV Editor was not optimized for the use cases. Customers also told us they needed to read this structured data in the time it takes to make an API call within workflows.

Today, we are happy to announce that we have created a new feature designed to quickly read and query large flat files: the CSV Reader. Our new CSV Reader will let customers rapidly query very large flat files with 100s of millions of rows and paginate through the results. We can even read through a 1-gigabyte flat file in seconds! Customers can also export all (or a portion) of the data into a new CSV file by applying a filter to the dataset just as quickly.

Now, you may be wondering: “When should I use the CSV Editor versus the CSV Reader?” Well, if you have a workflow that needs to manipulate any data within your flat file like changing specific values within cells, adding columns or rows, or joining multiple flat files together before moving it into another system, use our CSV Editor.

If you simply need to read or extract data from a flat file and move that data into another system, use the CSV Reader to do these more basic operations within your workflows. Another situation where you should use the CSV Reader is if you need to map or change field names from your initial flat file to align with your destination system’s field names.

We will be adding new capabilities to this feature, so stay tuned for more updates soon.

Have questions? Check out our new documentation platform today! Do you want to see the Tray Platform in action? Register for one of our weekly group demos.