This post serves as a follow up to my:
Converting Azure Firewall logs in JSON format created from Archive to a storage account diagnostic setting to CSV format
https://blog.terenceluk.com/2023/07/converting-azure-firewall-logs-in-json.html
… where I provided a script to convert a single JSON file that stored Azure Firewall Logs in a Storage Account container.
As noted in the previous post, I want to follow up with a script that would traverse through a folder reading JSON files in the sub-directories and converting them to CSVs to avoid manually generating each CSV for every hour of the day.
Additional reasons for using this script are:
- Allow the retrieval of archived Azure Firewall Logs that are no longer stored in Log Analytics
- Bulk converting JSON Line files to CSVs with a specified start and end date
- A method for working around the 30,000 records return limit when using Log Analytics Workspaces to query data
The entries for Azure Firewall Log activities can get fairly large so this script will read through each JSON file that are broken up in each hour of the day and convert them to CSV files. I originally thought about combining the files but working out the math for days of longs meant file sizes can get into the GBs and attempting to work with CSV files that large won’t be pleasant.
The JSON script can be found at GitHub repository here: https://github.com/terenceluk/Azure/blob/main/Azure%20Firewall/Bulk-Convert-Az-Firewall-Logs-JSON-to-CSV.ps1
The output file list would look as such:
Hope this helps anyone who may be looking for such a script that will save them the time required to manually convert the logs.
