Hey everyone, I'm trying to collect logs for work items cooked in Houdini Services and while I know where the logs end up I want a pythonic way of getting the log file location.
I saw this thread: https://www.sidefx.com/forum/topic/73446/ [www.sidefx.com]
But it's a few years old and I doubt it's talking about Services.
I know I can get the scheduler (for local cooking) in python and therefore the local log for a work item using pdg.Scheduler.getLogURI(work_item) but as far as I'm aware the scheduler isn't used by the Service at all.
Is there an equivalent for work_items in Services?
Thank you
PDG Services - Accessing Log from work_item
1014 7 1-
- AdamChabane
- Member
- 20 posts
- Joined: 12月 2020
- Offline
-
- AslakKS
- Member
- 184 posts
- Joined: 2月 2016
- Offline
Hi, having wanted this in the past I took a look.
In a python script this seems to work, to get the log paths (used a python script TOP, running on generate)
You would have to make sure that the services are actually running at that stage, I usually set these kinds of scripts to wait for all upstream items to be cooked.
In a python script this seems to work, to get the log paths (used a python script TOP, running on generate)
You would have to make sure that the services are actually running at that stage, I usually set these kinds of scripts to wait for all upstream items to be cooked.
sm = pdg.ServiceManager.get() service = sm.getService("blockTest") clients = service.clients logPaths = [] for clientName in clients: client = service.client(clients[0]) logPaths.append(client.logPath) print(logPaths)
Edited by AslakKS - 2024年5月27日 15:32:12
-
- AdamChabane
- Member
- 20 posts
- Joined: 12月 2020
- Offline
-
- AdamChabane
- Member
- 20 posts
- Joined: 12月 2020
- Offline
-
- AslakKS
- Member
- 184 posts
- Joined: 2月 2016
- Offline
I think you're correct, you would need to add marks to the log files or do something like this:
As long as you're running in a "Houdini Service Block" you can at the start of the loop store the current number of lines in the log.
Then at the end of the block extract the lines from the stored line number to the end of the file (need to reset the client make it write to the log).
Script at start of loop:
Script at end of loop (With the parm "Reset Service" set to "Reset Client" - "Before Cook"):
Attaching my mock-up scene
As long as you're running in a "Houdini Service Block" you can at the start of the loop store the current number of lines in the log.
Then at the end of the block extract the lines from the stored line number to the end of the file (need to reset the client make it write to the log).
Script at start of loop:
import os import sys import re command_string = " ".join(sys.argv) work_item.setStringAttrib("command_string", command_string) # Parse command string to extract --logfile {path} log_file_match = re.search(r"--logfile\s+(\S+)", command_string) if log_file_match: log_file = log_file_match.group(1) work_item.setStringAttrib("log_file", log_file) if os.path.isfile(log_file): with open(log_file) as f: current_line = sum(1 for _ in f) work_item.setIntAttrib("log_start", current_line) else: work_item.setIntAttrib("log_start", 1)
Script at end of loop (With the parm "Reset Service" set to "Reset Client" - "Before Cook"):
from itertools import islice log_start = work_item.intAttribValue("log_start") with open(work_item.stringAttribValue("log_file")) as f: lines = list(islice(f, log_start+2, None)) # Skip the first two lines work_item.setStringAttrib("log_contents",'\n'.join(lines))
Attaching my mock-up scene
Edited by AslakKS - 2024年5月27日 18:35:02
-
- tpetrick
- スタッフ
- 596 posts
- Joined: 5月 2014
- Offline
For tasks that run on services there isn't a separate log file per work item -- there's a single log for the whole service client. However, the individual log data for work items that ran on a service client is available through the pdg.WorkItem.logMessages property, which is basically just a string buffer containing the log data that was produced while that work item ran. That's also where log data is written for any in-process work items that call addMessage, addWarning or addError as part of their script code.
You can determine the name of the service client that ran a particular task using the pdg.WorkItemStats object returned from the pdg.WorkItem.stats(..) method. That object contains the various cook time durations for the work item, as well as the name of the service client that ran the work item if applicable:
You can determine the name of the service client that ran a particular task using the pdg.WorkItemStats object returned from the pdg.WorkItem.stats(..) method. That object contains the various cook time durations for the work item, as well as the name of the service client that ran the work item if applicable:
stats = work_item.stats() print("client = {}".format(stats.serviceClient)) print("service = {}".format(work_item.node.serviceName))
Edited by tpetrick - 2024年5月28日 16:16:24
-
- AslakKS
- Member
- 184 posts
- Joined: 2月 2016
- Offline
-
- AdamChabane
- Member
- 20 posts
- Joined: 12月 2020
- Offline
Fantastic!
Apologies I didn't realise I could reference an out-of-process work_item while still in a Service Block.
I am usingto get the in parent in-service work item and then from an in-process script, which is allowing me to pass thorough the logs without worrying about on disk log files.
Thank you both!
Apologies I didn't realise I could reference an out-of-process work_item while still in a Service Block.
I am using
pdg.WorkItem.parent
.logMessages
Thank you both!
-
- Quick Links