How to convert string list to tensor in pytorch?

Question

I'm working on making and predicting several models using phytorch. Because of memory problems, I made the tensor list into a data frame and saved it as Excel. After that, I tried to predict the model by loading the data stored in Excel, but when I called Excel, the tensor list changed to a str list. How can I change this str list back to the tensor list? I'll refer to part of the code, the original tensor.


def BERT_reasoning(tokens_tensor, segments_tensors):
    model.eval()
    predictions=[]
    for i in range(len(tokens_tensor)):
        if torch.cuda.is_available():
            tokens_tensor[i] = tokens_tensor[i].to('cuda')
            segments_tensors[i] = segments_tensors[i].to('cuda')
            model.to('cuda')
            with torch.no_grad():
                outputs = model(tokens_tensor[i], token_type_ids=segments_tensors[i])
                predictions.append(outputs[0])
                torch.cuda.empty_cache()
    return(predictions)


predictions=[0 for i in range(len(target))]
for i in tqdm(range(len(target))):
    predictions[0]=BERT_reasoning(tokens_tensor[i],segments_tensors[i])
    globals()['df_pred_{}'.format(i)]=pd.DataFrame(predictions[0])
    del predictions[0]
    excel_name='prediction_{}.xlsx'.format(i)
    globals()['df_pred_{}'.format(i)].to_excel(excel_name)
    del globals()['df_pred_{}'.format(i)]
    torch.cuda.empty_cache()



Result :
orginal tensor -
tensor([[[ -7.2395,  -7.2337,  -7.2301,  ...,  -6.6463,  -6.5081,  -4.4686],
         [ -8.1057,  -8.1946,  -8.0791,  ...,  -8.4518,  -7.6345,  -5.3930],
         [-10.7883, -10.6919, -10.5438,  ...,  -9.9607, -10.0536,  -6.8828],
         ...,
         [ -9.0698,  -9.3295,  -8.9949,  ...,  -6.1696,  -7.4357,  -7.4828],
         [ -6.3161,  -6.4182,  -6.5455,  ...,  -5.5366,  -5.7362,  -2.2207],
         [-12.0209, -11.9511, -12.0039,  ..., -11.8723,  -9.6545,  -8.2306]]],
       device='cuda:0')

changed str
"tensor([[[ -7.2395,  -7.2337,  -7.2301,  ...,  -6.6463,  -6.5081,  -4.4686],\n         [ -8.1057,  -8.1946,  -8.0791,  ...,  -8.4518,  -7.6345,  -5.3930],\n         [-10.7883, -10.6919, -10.5438,  ...,  -9.9607, -10.0536,  -6.8828],\n         ...,\n         [ -9.0698,  -9.3295,  -8.9949,  ...,  -6.1696,  -7.4357,  -7.4828],\n         [ -6.3161,  -6.4182,  -6.5455,  ...,  -5.5366,  -5.7362,  -2.2207],\n         [-12.0209, -11.9511, -12.0039,  ..., -11.8723,  -9.6545,  -8.2306]]],\n       device='cuda:0')"

If you open to store file in pickle format instead excel? - Use [Serialization - `torch.save`, `torch.load`](https://pytorch.org/docs/stable/torch.html#serialization) — Dishin H Goyani, Sep 18 '20 at 04:14
Does this answer your question? [Extract tensor from string](https://stackoverflow.com/questions/62814427/extract-tensor-from-string) — iacob, Mar 23 '21 at 15:52

score 0 · Answer 1 · answered Sep 17 '20 at 12:33

You can use the built-in eval function to obtain a tensor out of the string. Note that your tensor should not contain ellipsis (i.e '...') because the tensor won't be well-defined. All values should appear in the string you wish to recover (otherwise, there's no way to determine what they should be). Example:

t = eval("tensor([[1,2,3],[4,5,6]], device='cuda:0')")

How to convert string list to tensor in pytorch?

1 Answers1