How to Read Data from Standard Inputs in Pandas

Is it possible to read read_csv from the standard input (STDIN) on the pandas?

It is possible to put STDIN in a variable and set it to stringIO and read it, but is there a smarter way to do it?Ideally, I'd like to configure a stream (such as standard output) in a jupyter environment, and then load that stream into the pandas.

python pandas

2022-09-30 19:26

2 Answers

some PG|python readcsv.py
If you want to boot as shown in , do the following.

import sys
import pandas aspd

df = pd.read_csv(sys.stdin)
print(df)

If you type in jupyter, you can do it like this

import pandas as pd

stdin=input()
df = pd.read_csv(stdin)
print(df)

Add

You can run the sub-process on Jupiter to get the results as follows.

 variable = ! Some command

Therefore, CSV and JSON can be as follows (for UNIX series)

out=!echo-e "A, B, C\n10, 20, 30\n40, 50, 60"
outj=!ip-jaddr
import pandas aspd
importio
display(pd.read_csv(io.StringIO(out.n)))
display(pd.read_json(io.StringIO(outj.n)))

There is also a way to capture the target cell (% capture result), but StringIO is used the same

Add More

If you use PIPE and sub-process, you don't have to pay for it, but you need code

from subprocess import Popen,PIPE
cmd = 'ip-jaddr'.split('')
with Popen(cmd, stdout=PIPE, stderr=PIPE, text=True) as proc:
    df = pd.read_json(proc.stdout)
display(df)

2022-09-30 19:26

This article deals with that.
[python] io.StringIO is convenient, so you can use it well

importio
import pandas aspd

US>"txt=""
number, name, score
1, hoge, 100
2, fuga, 200
3,piyo,300
"""

df=pd.read_csv(io.StringIO(txt), index_col="number")
print(df)

The other reading of json is sys.stdin directly specified in this article.
The article was written in the 2.x series printdf, so in the quotation below, we changed it to the 3.x series print(df).
Pandas data from stdin

import sys
import pandas aspd

df = pd.read_json(sys.stdin)
print(df)

Changing read_json to read_csv worked fine.

And this is how to read the data output from subprocess by converting it to stringIO.
I haven't tried it because it's an example on Unix, but it will work because it's approved.
Input for read_csv pandas function

importio
import subprocess
import pandas

cmd = ('cat', '/tmp/csvfile')
process=subprocess.Popen (cmd, stdout=subprocess.PIPE)
csv=io.StringIO()
for line in process.stdout:
    csv.write(line.decode().trip('"\n')+'\n')
csv.seek(0)
data=pandas.read_csv(csv,index_col=0)
csv.close()

2022-09-30 19:26

If you have any answers or tips

Popular Tags

python x 4647

android x 1593

java x 1494

javascript x 1427

c x 927

c++ x 878

ruby-on-rails x 696

php x 692

python3 x 685

html x 656