Am writing this post because had such a problem and it was so easy to fix so hopefully will be useful to others and save you spending hours yourself!
I was exporting from mysql database in PHPmyAdmin to a csv file. However, when I opened it up in Excel, all of the Hebrew was gibberish.
Instead, I opened it up in Google Sheets via the web and all looking good now.
Simple but effective 🙂