Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qct-parse can have multiple inputs #34

Merged
merged 2 commits into from
Dec 4, 2024
Merged

qct-parse can have multiple inputs #34

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b7f57a8
split main function into get_arg_parser and parse_qc_tools_report fun…
henryborchers Dec 3, 2024
c568e84
qct-parse can have multiple inputs
henryborchers Dec 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
172 changes: 97 additions & 75 deletions qct_parse/qct_parse.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,21 @@ def initLog(inputPath):
logPath = inputPath + '.log'
logging.basicConfig(filename=logPath,level=logging.INFO,format='%(asctime)s %(message)s')
logging.info("Started QCT-Parse")


def set_logger(input_path):
log_path = f'{input_path}.log'
logger = logging.getLogger()
if logger.hasHandlers():
logger.handlers.clear()
logger.setLevel(logging.INFO)

file_handler = logging.FileHandler(filename=log_path)
file_handler.setLevel(logging.INFO)
file_handler.setFormatter(logging.Formatter('%(asctime)s %(message)s'))
logger.addHandler(file_handler)
logger.info("Started QCT-Parse")



# finds stuff over/under threshold
def threshFinder(inFrame,args,startObj,pkt,tag,over,thumbPath,thumbDelay,adhoc_tag):
Expand Down Expand Up @@ -613,51 +627,9 @@ def color_percentage(value):
print(f"Frames With At Least One Fail:\t{overallFrameFail}\t{color}{percent_overall_string}{RESET}\t% of the total # of frames\n")
print(f"{BOLD}**************************{RESET}\n")


def main():
"""
Main function that parses QCTools XML files, applies analysis, and optionally exports thumbnails.

This function handles command-line arguments to process a QCTools report, extract frame data from the XML,
apply threshold analysis for broadcast values, optionally detect color bars, and export analysis results to
the console or thumbnails.

Command-line Arguments:
-i, --input (str): Path to the input QCTools XML.gz file.
-t, --tagname (str): Tag name to analyze, e.g., SATMAX.
-o, --over (float): Overage threshold for the tag specified.
-u, --under (float): Under threshold for the tag specified.
-p, --profile (str): Profile or template name from the qct-parse_config.txt file, e.g., 'default'.
-buff, --buffSize (int): Circular buffer size. Defaults to 11, ensures odd number.
-te, --thumbExport: Export thumbnails if flag is set.
-ted, --thumbExportDelay (int): Minimum number of frames between exported thumbnails.
-tep, --thumbExportPath (str): Path to export thumbnails, defaults to input basename if not provided.
-ds, --durationStart (float): Start time in seconds for analysis.
-de, --durationEnd (float): End time in seconds for analysis.
-bd, --barsDetection: Flag to enable color bars detection.
-pr, --print: Flag to print frame data to the console.
-q, --quiet: Hide ffmpeg output if flag is set.

Workflow:
1. Parse command-line arguments.
2. Optionally load reference threshold values from a profile in `qct-parse_config.txt`.
3. Initialize buffers, frame counters, and paths for thumbnail export.
4. Check for `pkt_dts_time` or `pkt_pts_time` in the QCTools XML file.
5. Set the analysis duration start and end times.
6. Perform bars detection if enabled, otherwise proceed with general analysis.
7. Call the `analyzeIt` function to perform frame-by-frame analysis and calculate exceedances.
8. Print results using `printresults` if applicable.
9. Handle errors or invalid input (e.g., missing thumbnail export flag but specifying a path).

Example usage:
python qct-parse.py -i sample.qctools.xml.gz -t SATMAX -o 5.0 -u -5.0 -te

Returns:
None: The function processes the XML file, performs analysis, and optionally exports thumbnails and prints results to the console.
"""
#### init the stuff from the cli ########
def get_arg_parser():
parser = argparse.ArgumentParser(description="parses QCTools XML files for frames beyond broadcast values")
parser.add_argument('-i','--input',dest='i', help="the path to the input qctools.xml.gz file")
parser.add_argument('-i','--input',dest='i', action='append', help="the path to the input qctools.xml.gz file")
parser.add_argument('-t','--tagname',dest='t', help="the tag name you want to test, e.g. SATMAX")
parser.add_argument('-o','--over',dest='o', help="the threshold overage number")
parser.add_argument('-u','--under',dest='u', help="the threshold under number")
Expand All @@ -672,28 +644,23 @@ def main():
parser.add_argument('-be','--barsEvaluation',dest='be',action ='store_true',default=False, help="turns Color Bar Evaluation on and off")
parser.add_argument('-pr','--print',dest='pr',action='store_true',default=False, help="print over/under frame data to console window")
parser.add_argument('-q','--quiet',dest='q',action='store_true',default=False, help="hide ffmpeg output from console window")
args = parser.parse_args()

## Validate required arguments
if not args.i:
parser.error("the following arguments are required: -i/--input [path to QCTools report]")
if args.o and args.u:
parser.error("Both the -o and -u options were used. Cannot set threshold for both over and under, only one at a time.")

##### Initialize variables and buffers ######
startObj = args.i.replace("\\","/")
return parser


def parse_single_qc_tools_report(input_file, args):
startObj = input_file.replace("\\","/")
extension = os.path.splitext(startObj)[1]
# If qctools report is in an MKV attachment, extract .qctools.xml.gz report
# If qctools report is in an MKV attachment, extract .qctools.xml.gz report
if extension.lower().endswith('mkv'):
startObj = extract_report_mkv(startObj)
buffSize = int(args.buff) # cast the input buffer as an integer
if buffSize%2 == 0:
buffSize = buffSize + 1
initLog(startObj) # initialize the log
set_logger(startObj)
overcount = 0 # init count of overs
undercount = 0 # init count of unders
count = 0 # init total frames counter
framesList = collections.deque(maxlen=buffSize) # init framesList
framesList = collections.deque(maxlen=buffSize) # init framesList
thumbDelay = int(args.ted) # get a seconds number for the delay in the original file btw exporting tags
parentDir = os.path.dirname(startObj)
baseName = os.path.basename(startObj)
Expand All @@ -702,7 +669,7 @@ def main():
durationEnd = args.de

# we gotta find out if the qctools report has pkt_dts_time or pkt_pts_time ugh
with gzip.open(startObj) as xml:
with gzip.open(startObj) as xml:
for event, elem in etree.iterparse(xml, events=('end',), tag='frame'): # iterparse the xml doc
if elem.attrib['media_type'] == "video": # get just the video frames
# we gotta find out if the qctools report has pkt_dts_time or pkt_pts_time ugh
Expand All @@ -712,10 +679,10 @@ def main():
break

###### Initialize values from the Config Parser
# Determine if video values are 10 bit depth
# Determine if video values are 10 bit depth
bit_depth_10 = detectBitdepth(startObj,pkt,framesList,buffSize)
# init a dictionary where we'll store reference values from our config file
profile = {}
profile = {}
# init a list of every tag available in a QCTools Report
tagList = ["YMIN","YLOW","YAVG","YHIGH","YMAX","UMIN","ULOW","UAVG","UHIGH","UMAX","VMIN","VLOW","VAVG","VHIGH","VMAX","SATMIN","SATLOW","SATAVG","SATHIGH","SATMAX","HUEMED","HUEAVG","YDIF","UDIF","VDIF","TOUT","VREP","BRNG","mse_y","mse_u","mse_v","mse_avg","psnr_y","psnr_u","psnr_v","psnr_avg"]

Expand All @@ -727,13 +694,13 @@ def main():
durationStart = float(args.ds) # The duration at which we start analyzing the file if no bar detection is selected
elif not args.de == 99999999:
durationEnd = float(args.de) # The duration at which we stop analyzing the file if no bar detection is selected
# set the path for the thumbnail export


# set the path for the thumbnail export
if args.tep and not args.te:
print("Buddy, you specified a thumbnail export path without specifying that you wanted to export the thumbnails. Please either add '-te' to your cli call or delete '-tep [path]'")
exit()

if args.tep: # if user supplied thumbExportPath, use that
thumbPath = str(args.tep)
else:
Expand All @@ -744,12 +711,12 @@ def main():
thumbPath = os.path.join(parentDir, str(args.t) + "." + str(args.u))
else: # if they're using a profile, put all thumbs in 1 dir
thumbPath = os.path.join(parentDir, "ThumbExports")

if args.te: # make the thumb export path if it doesn't already exist
if not os.path.exists(thumbPath):
os.makedirs(thumbPath)


######## Iterate Through the XML for Bars detection ########
if args.bd:
print(f"\nStarting Bars Detection on {baseName}\n")
Expand All @@ -770,7 +737,7 @@ def main():
else:
durationStart = ""
durationEnd = ""

if args.p is not None:
# create list of profiles
list_of_templates = args.p
Expand All @@ -791,7 +758,7 @@ def main():
if not config.has_section(template):
print(f"Profile '{template}' does not match any section in the config.")
continue # Skip to the next template if section doesn't exist
for t in tagList: # loop thru every tag available and
for t in tagList: # loop thru every tag available and
try: # see if it's in the config section
profile[t.replace("_",".")] = config.get(template,t) # if it is, replace _ necessary for config file with . which xml attributes use, assign the value in config
except: # if no config tag exists, do nothing so we can move faster
Expand All @@ -801,8 +768,8 @@ def main():
print(f"\nStarting Analysis on {baseName} using assigned profile {template}\n")
kbeyond, frameCount, overallFrameFail = analyzeIt(args,profile,startObj,pkt,durationStart,durationEnd,thumbPath,thumbDelay,framesList,adhoc_tag=False)
printresults(kbeyond,frameCount,overallFrameFail)
if args.t and args.o or args.u:

if args.t and args.o or args.u:
profile = {}
tag = args.t
if args.o:
Expand All @@ -813,10 +780,65 @@ def main():
print(f"\nStarting Analysis on {baseName} using user specified tag {tag} w/ threshold {over}\n")
kbeyond, frameCount, overallFrameFail = analyzeIt(args,profile,startObj,pkt,durationStart,durationEnd,thumbPath,thumbDelay,framesList,adhoc_tag = True)
printresults(kbeyond,frameCount,overallFrameFail)

print(f"\nFinished Processing File: {baseName}.qctools.xml.gz\n")

return

def parse_qc_tools_report(args):
##### Initialize variables and buffers ######
for input_file in args.i:
parse_single_qc_tools_report(input_file, args)

def main():
"""
Main function that parses QCTools XML files, applies analysis, and optionally exports thumbnails.

This function handles command-line arguments to process a QCTools report, extract frame data from the XML,
apply threshold analysis for broadcast values, optionally detect color bars, and export analysis results to
the console or thumbnails.

Command-line Arguments:
-i, --input (str): Path to the input QCTools XML.gz file.
-t, --tagname (str): Tag name to analyze, e.g., SATMAX.
-o, --over (float): Overage threshold for the tag specified.
-u, --under (float): Under threshold for the tag specified.
-p, --profile (str): Profile or template name from the qct-parse_config.txt file, e.g., 'default'.
-buff, --buffSize (int): Circular buffer size. Defaults to 11, ensures odd number.
-te, --thumbExport: Export thumbnails if flag is set.
-ted, --thumbExportDelay (int): Minimum number of frames between exported thumbnails.
-tep, --thumbExportPath (str): Path to export thumbnails, defaults to input basename if not provided.
-ds, --durationStart (float): Start time in seconds for analysis.
-de, --durationEnd (float): End time in seconds for analysis.
-bd, --barsDetection: Flag to enable color bars detection.
-pr, --print: Flag to print frame data to the console.
-q, --quiet: Hide ffmpeg output if flag is set.

Workflow:
1. Parse command-line arguments.
2. Optionally load reference threshold values from a profile in `qct-parse_config.txt`.
3. Initialize buffers, frame counters, and paths for thumbnail export.
4. Check for `pkt_dts_time` or `pkt_pts_time` in the QCTools XML file.
5. Set the analysis duration start and end times.
6. Perform bars detection if enabled, otherwise proceed with general analysis.
7. Call the `analyzeIt` function to perform frame-by-frame analysis and calculate exceedances.
8. Print results using `printresults` if applicable.
9. Handle errors or invalid input (e.g., missing thumbnail export flag but specifying a path).

Example usage:
python qct-parse.py -i sample.qctools.xml.gz -t SATMAX -o 5.0 -u -5.0 -te

Returns:
None: The function processes the XML file, performs analysis, and optionally exports thumbnails and prints results to the console.
"""
#### init the stuff from the cli ########
parser = get_arg_parser()
args = parser.parse_args()
## Validate required arguments
if not args.i:
parser.error("the following arguments are required: -i/--input [path to QCTools report]")
if args.o and args.u:
parser.error("Both the -o and -u options were used. Cannot set threshold for both over and under, only one at a time.")
parse_qc_tools_report(args)


if __name__ == '__main__':
dependencies()
Expand Down
Loading