Working with CSV files with the NumPy module

The NumPy module provides two functions for reading values from CSV files: np.loadtxt() and np.genfromtxt().

An example of np.loadtxt is as follows:

arr = np.loadtxt('temp.csv', skiprows=1, usecols=(2,3), delimiter=',')
arr

The preceding code reads columns 3 and 4 from the file that we created earlier, and saves them in a 9 × 2 array as follows:

array([[2.58 , 0.136],
       [2.552, 0.1  ],
       [2.55 , 0.1  ],
       [2.55 , 0.1  ],
       [2.554, 0.1  ],
       [2.55 , 0.1  ],
       [2.534, 0.096],
       [2.484, 0.   ],
       [2.468, 0.   ]])

The np.loadtxt() function cannot handle CSV files with missing data. For instances where data is missing, np.genfromtxt() can be used. Both of these functions offer many more arguments; details can be found in the NumPy documentation. The preceding code can be written using np.genfromtxt() as follows:

arr = np.genfromtxt('temp.csv', skip_header=1, usecols=(2,3), delimiter=',')

NumPy arrays produced as a result of applying AI to IoT data can be saved with np.savetxt(). For example, the array we loaded previously can be saved as follows:

np.savetxt('temp.csv', arr, delimiter=',')

The np.savetxt() function also accepts various other useful arguments, such as the format for saved fields and headers. Check the NumPy documentation for more details on this function.

CSV is the most popular data format on IoT platforms and devices. In this section, we learned how to read CSV data using three different packages in Python. Let's learn about XLSX, another popular format, in the next section.