RC4 encryption cpp algorithm

Question

I am writing a program that can encrypt files using the cipher RC4, In my file "test.txt" is a word "Plaintext", the program should encrypt and save it also in a file (for test I was using "cout")

I thing there is a problem with the last part of the code

while ( plik.read(&x,1) )
{
  i = ( i + 1 ) % 256;
  j = ( j + S [ i ] ) % 256;
  swap( S [ i ], S [ j ] );
  temp = S [ ( S [ i ] + S [ j ] ) % 256 ] ^ x;
  cout << temp;
} 

There is no errors and no warnings, What should I change?

#include <algorithm>
#include <fstream>
#include <iostream>
#include <string>

using namespace std;


int main()
{
 
 unsigned char S[256];

 int i = 0;
 for ( i = 0; i < 256; i++ )
   S [ i ] = i;

 string Key;
 cout << "Enter the key: ";
 cin >> Key;

int j = 0;
 for ( i = 0; i < 256; i++ )
 {
   j = ( j + S [ i ] + Key.at( i % Key.length() ) ) % 256;
   swap( S [ i ], S [ j ] );
 }

 ifstream plik; 
 string path = "tekst.txt";
 plik.open( path );
 char tekst;
 plik >> tekst;

 string printFile = "Encryption_" + path;
 ofstream outfile( printFile );
 
 char x;
 j = 0;
 i = 0;
 string temp;
 while ( plik.read(&x,1) )
 {
   i = ( i + 1 ) % 256;
   j = ( j + S [ i ] ) % 256;
   swap( S [ i ], S [ j ] );
   temp = S [ ( S [ i ] + S [ j ] ) % 256 ] ^ x;
   cout << temp;
 }
 
 plik.close();
 outfile.close();
 
 return 0;
}

My key is: "key" and the result should be

Good Result

Answer 1

Your algorithm is mathematically totally correct, I just did few small corrections to other parts of your code and your code made correct output.

For testing purposes lets take test examples here from Wiki, first example. Input key will be Key, input file will be Plaintext, resulting file should be BB F3 16 E8 D9 40 AF 0A D3 in hex. Following modified code passes this test.

In code below you input a key from console, then input file is taken from tekst.txt and output file is written to Encryption_tekst.txt. It is better not to print result to console as ASCII chars but to view resulting file in hex viewer, because console will mess up with character encodings, better to view in hex.

I added printing to console of resulting hex characters. Also notice that in following code very first block writes Plaintext to tekst.txt, I did this for the purpose of example, so that code can be just copy pasted and run without any extra dependencies. You have to delete very first block that writes to teks.txt, because you have your own input tekst.txt and it will be overwritten.

When you run following example enter Key in console prompt.

Try it online!

#include <algorithm>
#include <fstream>
#include <iostream>
#include <iomanip>
#include <string>
#include <cstdint>

using namespace std;

int main() {
    {
        ofstream out("tekst.txt");
        out << "Plaintext";
    }

    unsigned char S[256];

    int i = 0;
    for (i = 0; i < 256; i++)
        S[i] = i;

    string Key;
    cout << "Enter the key: ";
    cin >> Key;

    int j = 0;
    for (i = 0; i < 256; i++) {
        j = (j + S[i] + Key.at(i % Key.length())) % 256;
        swap(S[i], S[j]);
    }

    ifstream plik;
    string path = "tekst.txt";
    plik.open(path);

    string printFile = "Encryption_" + path;
    ofstream outfile(printFile);

    char x;
    j = 0;
    i = 0;
    char temp = 0;
    while (plik.read(&x, 1)) {
        i = (i + 1) % 256;
        j = (j + S[i]) % 256;
        swap(S[i], S[j]);
        temp = S[(S[i] + S[j]) % 256] ^ x;
        outfile << temp;
        cout << std::hex << std::setw(2) << std::setfill('0')
             << uint32_t(uint8_t(temp)) << " ";
    }

    plik.close();
    outfile.close();

    return 0;
}

Input key (from console):

Key

Input file tekst.txt:

Plaintext

Console output:

Enter the key: Key
bb f3 16 e8 d9 40 af 0a d3

Output Encryption_tekst.txt viewed in hex viewer with code page CP1252: