IBM Bob on Netcool code
Five prompts that take a Netcool Impact policy and a set of Netcool probe rules from a defect register to fixed, cleaned, standardized and reviewed code, with IBM Bob in its built-in Agent mode.
The use case
Use case 5 of the pilot is code analysis, standardization and optimization for Netcool: Impact policies and probe rules that grew over years, with defects nobody has listed, commented-out code left in place, hard-coded paths and inconsistent names. The client sent two pieces of production code, each with its own document saying what the code should become:
An Impact policy
About 3,300 lines that build an incident ticket from an alarm and send it on. Its guide is the client's refactoring requirement document: defects to fix, logging and dead code to tidy, camelCase naming, and a three-tier modular structure.
Probe rules
About 1,900 lines of rules for one element manager, with ten lookup files. Its guide is the client's code standards document: no hard-coded values, paths through an environment variable, one log format, no commented-out code, and a standard directory tree.
Nothing here runs on an Impact server or on a probe. Everything the prompts claim is shown from the code alone, and every report says what still needs a test where the code runs.
The input data, and why each is used
The client sent production code, not samples: an Impact policy and a probe rules file that run today, each with the client's own document saying what the code should become. That pairing is the point of the use case. The code shows how the system works now, with everything years of changes left in it; the document is the client's own definition of "done", so every change Bob makes can be checked against something the client wrote rather than against Bob's taste.
| Input | What it is | What is in it | Why this input is used |
|---|---|---|---|
| The Impact policy | One production Netcool Impact policy (IPL), about 3,300 lines. | The whole path from an alarm flagged for a ticket to the incident sent to the ticketing system: it checks whether the alarm may raise a ticket, turns the alarm's times into local dates, sets the ticket's fields (requester, severity, impact, urgency, service level), builds the ticket's title and work log with formatting rules per network domain and per alarm type, assembles an XML envelope, sends it asynchronously, and marks the alarm's ticket state in the event store. Over the years new domains and alarm types were added in place, so it holds repeated branches, commented-out blocks and mixed naming. | It is the code to analyse and improve: all five prompts work on it. A ticketing policy is a good test because a defect in it shows up far away, as a wrong or empty field in a ticket, and because it is long enough that nobody re-reads it whole by hand. |
| The policy's refactoring requirement | The client's document, about 430 lines of text. | What the policy does, in four core functions; then the refactoring agenda: defects to fix (variable case mismatches, misspelt event fields, malformed XML entities), database lookups to merge, a logging standard (every line prefixed with the event's serial, no noisy field dumps, every state change logged), dead code to remove, camelCase names that reveal intent, and a three-tier modular structure (an orchestrator, policies per network domain, policies per alarm type) with its target folder tree. | It is what Bob's work on the policy is checked against, in three ways. 1. In prompt 1, each defect is described by what it does to the ticket, using the document's description of the ticket (for example: "the ticket's severity field stays empty"). 2. Prompts 3 and 4 do two items of the document's section 3: prompt 3 removes the commented-out code, prompt 4 renames the variables to camelCase. 3. At the end of each step, the skill's check lists every requirement of the document that the step concerns; the step cannot pass until Bob marks each one as done, or as left to the client with a reason. Using the client's own document keeps the work tied to what the client asked for. |
| The probe rules | One production rules file for the probe that receives the alarms of one network element manager (an IP VPN manager), about 1,900 lines. | Ten table declarations with absolute paths, the classification of alarm types, the mapping of each alarm to the event's fields (node, key, group, severity, type, summary), enrichment from lookup tables, flags for automatic tickets, filters that discard noise, and code switched off during past changes. | A second, different kind of Netcool code, so the same five prompts are shown to work beyond one language: probe rules decide what every alarm looks like in the event list, so a defect here changes events rather than tickets. |
| The ten lookup and include files | Data and small rule files the rules read, from under 100 bytes to about 540 KB each. | Alarm names, pre-classification of alarm types, enrichment by node and port, automatic-ticket rules, node lists, the alarms included in service-level monitoring. | Context, not code to change. Bob reads them when a finding depends on them (a table used but not declared, a key that cannot match); the commands are given the rules file only, so the lookups are not "reviewed" as code. The path standardization changes the lines that point to them, never the files themselves. |
| The rules code standards | The client's document, about 55 lines. | Five standards: no hard-coded values (lists of nodes, ports or alarm identifiers moved into lookups), every path through $NC_RULES_HOME and all table declarations in one place, one key=value log format that names the rules file and the node, no commented-out code, and a standard folder tree for a probe's rules. | It is what Bob's work on the rules is checked against. Prompt 3 does standard 4 (remove the commented-out code). Prompt 4 does standard 2 (write every path through $NC_RULES_HOME). Standards 1, 3 and 5 need new code to be written, so they were kept out of the five prompts (see below); the new-integration use case writes new rules directly in the folder tree of standard 5. |
What was not sent, and why that is fine here. No running Impact server or probe, no sample alarms or tickets, no event-store schema. So every prompt claims only what the code itself shows, and every report ends with what still needs a test where the code runs. Both bodies of code sit in one Bob workspace, each in its own folder with its document, so the skill uses the policy's requirement for the policy and the rules' standards for the rules.
From input to output
| Prompt | Reads | Writes |
|---|---|---|
| 1 · Defect register | Each original file and its document; the lookups when a finding needs them | A defect register per input |
| 2 · Fix | Each original file and its register | A corrected copy (step 1) and a change log per input |
| 3 · Clean up | The step-1 copy and the document's item on commented-out code | A cleaned copy (step 2) and a clean-up report per input |
| 4 · Standardize | The step-2 copy and the document's item on names (policy) or paths (rules) | A standardized copy (step 3) and a standards report per input |
| 5 · Code review | The step-3 copy, the original, and every earlier report | A review of the whole change per input |
How it works
Bob is opened on one workspace folder that holds both inputs, each in its own folder with its document. Two small things in that folder steer it:
One skill: code-review
Instructions for five tasks (review code, fix, change existing code, review a change, diagnose) and the code that does the mechanical work. The skill is generic: it names no client, product or language.
A short rule
Tells Bob to use the skill before any review or change, to run its commands from the workspace root, and to check where the work stands before every reply.
The skill does the work, Bob does the judging
Each task is one command, run twice or three times. The first run scans the code and writes a notes file with everything Bob has to decide. Bob reads the code and writes its judgements in that notes file: which scan hits are real defects, which corrections to make, what the client must decide. The next run applies the notes, writes the report and the changed copy, and compares the new version with the old one. The skill writes every report and every changed file; Bob never types code or reports by hand.
Gates
Every command ends with a line that starts Gate:. A task is finished only when it prints Gate: passed. A gate refuses, for example:
- a review where a scan hit was left undecided, or a quoted line is not on the line it cites;
- a fix that leaves a High finding with no correction and no specific question for the client;
- a change that adds a call, an operator or a non-ASCII character the original did not have, unless the reason is written down;
- a change that leaves a file with half a block, or a requirement of the client's document unaccounted for;
- a review of a change where any difference between the two versions has not been judged.
A gate that passed means the work is complete and accounted for. It does not mean the code is right; that is what the reports and the last prompt are for.
One chat for everything
All five prompts go in one chat. Long chats get summarized by Bob on the way; a status command lets Bob see, from the files, which tasks are finished and which request it is answering, so it picks up where it was.
The five prompts
Each prompt does its step for the policy and then for the rules. Each step works on the output of the one before; the original files are never changed. The texts below are the prompts as sent, with the client's name replaced by "the client" and its file names by <policy>.ipl and <rules>.rules. Each prompt has three parts: the verb of the task and what it works on, Write and the report, and Do not and the limit; the verb tells Bob which task of the skill to run.
Bob scans and reads each file, decides each scan hit, adds what reading finds, and writes a register: every defect with its line, the quoted code, its effect on the ticket or the event, the fix, and how sure it is.
Review impact-policy/inputs/<policy>.ipl and probe-rules/inputs/<rules>.rules for defects, one after the other, each with the document from the client next to it as the guide. Write impact-policy/reports/defect-register.md and probe-rules/reports/defect-register.md, with the effect of each defect on the ticket for the policy and on the event for the rules. Do not change the policy or the rules file.
| Why this step | Before anything changes, list what is wrong and what each defect does to a ticket or an event. The client sees the size of the problem, and every later step refers back to this list. |
|---|---|
| Reads | The original policy and rules file, each with its document; the lookup files when a finding depends on them. |
| The skill | Scan the code for 41 kinds of defect (a name misspelt, read before it is set or never used; a single = where a comparison is meant; an XML entity or tag not closed; brackets that do not balance; branches that repeat each other; code that cannot run; a log that says the code discards or skips while the statement that would do it is switched off; a dashboard row that names another item), say for each hit where its value goes, and write the notes file with every hit to decide. |
| Bob decides | Whether each hit is a real defect, with a reason the workspace shows; by reading the whole file once, the defects the scan cannot see (see below); for each finding its effect, the fix and how sure it is; the questions only the client can answer. |
| The gate refuses | A hit left undecided; a finding whose quoted words are not on the line it cites; a "not a defect" without the evidence that settles it. |
| Writes | A defect register per input, with a notes file next to it that holds Bob's judgements. |
| Why these words | "The effect on the ticket / on the event" makes the register speak the client's language rather than the code's; "Do not change" keeps the step read-only. |
The limits. Reading is Bob's judgement, as a human reviewer's is: nothing can prove that every defect was found, so a person should read the findings from reading before they go to the client. Some defects cannot be seen in the code at all (a wrong value in a lookup table, a difference from the live system), which is why every register ends with Not checked: nothing was run on an Impact server or a probe.
The skill makes every correction that has only one possible form (a single = in a condition, a name missing its prefix, a misspelt field). Bob corrects what the code shows to be right and leaves every real choice to the client as a specific question. A correction that would hide a defect is held back as a proposal.
Fix the defects of impact-policy/inputs/<policy>.ipl in a copy under impact-policy/refactored/step-1-fixed and of probe-rules/inputs/<rules>.rules in a copy under probe-rules/refactored/step-1-fixed, one after the other. Write impact-policy/reports/change-log.md and probe-rules/reports/change-log.md. Do not decide what is the client's to decide.
| Why this step | Correct what is plainly wrong, in a copy, so that the original stays as the baseline and every change is written down. |
|---|---|
| Reads | The original file and its defect register. |
| The skill | Make every correction that has only one possible form, apply Bob's corrections, and compare the copy with the original. A correction tagged with the wrong finding is refused; switching commented-out code back on becomes a question, not a fix. |
| Bob decides | The correction of each finding the code itself settles, and a specific question for each one only the client can settle (a filter that was switched off on purpose, an address written into the code). |
| The gate refuses | A High finding with neither a correction nor a specific question; a new call, operator or non-ASCII character the original did not have, unless its reason is written; a conditional operator (?) the code does not already use; notes left untouched. A correction that writes a different number of lines than it replaces, or takes out more than two lines, is not made: it is listed for the operator as a proposal, with its text. |
| Writes | The corrected copy (step 1) and a change log per input. |
| Why these words | "Do not decide what is the client's to decide" turns business choices into questions instead of silent changes. |
The skill lists the commented-out code (lines switched off, disabled statements, whole blocks); Bob decides what goes, and the skill removes it in a copy. The events and tickets stay the same.
Clean up a copy of each step-1-fixed, one after the other. Policy: in a copy of impact-policy/refactored/step-1-fixed under impact-policy/refactored/step-2-cleaned, remove the commented-out code as section 3 of the requirement document asks. Rules: in a copy of probe-rules/refactored/step-1-fixed under probe-rules/refactored/step-2-cleaned, remove the commented-out code as standard 4 of the code standards document asks. Write impact-policy/reports/clean-up.md and probe-rules/reports/clean-up.md. Do not change the ticket content or the events.
| Why this step | Remove the code switched off over the years, so that engineers read only what runs. Both documents ask for it, and it is the change most often done carelessly by hand. |
|---|---|
| Reads | The step-1 copy and the document's item on commented-out code. |
| The skill | List every commented-out line and block with its text, telling them apart from comments that explain live code, and the comment lines right above them that may only label them; remove them all with one entry of the plan, less the lines Bob keeps, so Bob neither reads the file again nor types line numbers; work in a copy; refuse to remove a comment that stands right above a live line that stays; compare the copy with step 1. |
| Bob decides | Which listed lines stay (a comment that explains, not code), and which labels go with the code they mark, such as "# disabled by ops"; a heading stays. |
| The gate refuses | A removal that changes a live line or leaves half a block; an item of the document left unaccounted for. |
| Writes | The cleaned copy (step 2) and a clean-up report per input. |
| Why these words | "Do not change the ticket content or the events" states the safety condition the comparison checks. |
Policy: every local variable renamed to camelCase from the skill's proposals, except names that another component may read or that may come from outside. Rules: the absolute path prefix replaced by an environment variable with one rule, so that nothing else in those lines can change.
Standardize a copy of each step-2-cleaned, one after the other. Policy: in a copy of impact-policy/refactored/step-2-cleaned under impact-policy/refactored/step-3-standardized, rename the variables to camelCase as section 3 of the requirement document asks. Rules: in a copy of probe-rules/refactored/step-2-cleaned under probe-rules/refactored/step-3-standardized, replace the absolute path prefix /opt/IBM/tivoli/netcool/omnibus/etc/probes/rules with $NC_RULES_HOME in every include and table line, as standard 2 of the code standards document asks. Write impact-policy/reports/standards.md and probe-rules/reports/standards.md. Do not change the ticket content or the events, or any folder or file name after the path prefix.
| Why this step | Names that say what the data is, and paths that move with the installation: the two standardizations the documents ask for that the skill can make reliably. |
|---|---|
| Reads | The step-2 copy and the document's item on names (policy) or paths (rules). |
| The skill | Propose a camelCase name for every local variable and apply the map; refuse to rename a name that another component may read (event fields, names read before they are set, names that may come from outside); for the rules, replace the prefix with one rule on the include and table lines only. |
| Bob decides | Which proposed names to take, which to adjust, and which to leave for the client to confirm. |
| The gate refuses | A rename that merges two names or touches an event field; a path line where anything other than the prefix changed. |
| Writes | The standardized copy (step 3) and a standards report per input. |
| Why these words | The exact prefix and "any folder or file name after the path prefix" were added after a run where a looser prompt also changed folder names. |
The skill compares the final copy with the original and writes the review: both versions in numbers, the defect counts before and after, what the change consists of line by line, and every difference for Bob to judge as no change of behaviour, intended or unintended. Bob adds what reading finds, checks the claims of the earlier reports, and says what only the client can settle.
Review impact-policy/refactored/step-3-standardized against impact-policy/inputs/<policy>.ipl, and probe-rules/refactored/step-3-standardized against probe-rules/inputs/<rules>.rules, one after the other. Write impact-policy/reports/code-review.md and probe-rules/reports/code-review.md. Do not fix anything.
| Why this step | An independent check of the final copy against the original: everything that changed, judged, before anyone deploys it. |
|---|---|
| Reads | The final copy, the original, and every earlier report and notes file. |
| The skill | Compare the two versions line by line, count the defects before and after, group the differences (corrections, removals, renames, path changes), and check the claims of the earlier reports. |
| Bob decides | For each difference: no change of behaviour, intended (and by which report), or unintended; what reading adds; the client's open decisions as questions. |
| The gate refuses | Any difference left unjudged; a claim of an earlier report that the files do not bear out. |
| Writes | A review of the whole change per input. |
| Why these words | "Do not fix anything" keeps the reviewer and the author apart, even when both are Bob. |
What the rehearsals showed
The prompts were rehearsed in Bob from a clean workspace, with nothing typed in the chat but the five prompts, and every step was checked by hand against the original before the next prompt was sent. A step counts as right only when its gates passed and the changed copy held when read line by line.
| Run | What happened | What changed after it |
|---|---|---|
| First run, full scope | Register and fix usable. The clean-up needed three follow-up messages. The split into modules passed its gate and was wrong for both inputs: blocks cut in two between files, the order of running changed, files left behind. | A fix can no longer pass with its notes untouched; a change must account for each requirement of the client's document; a file left with half a block cannot be explained away. The split, the log rewrite and the lookup files were taken out of the five prompts. |
| Clean run 1 | Prompts 1 to 4 passed their gates at the first attempt. Checked by hand, the rules standardization had changed folder names while replacing the path prefix and left some absolute paths. | Prompt 4 asks the rules for the prefix alone, done by one rule, so that nothing else in those lines can change. |
| Clean run 2 | All ten gates passed at the first attempt, about 30 minutes and 30 Bob coins, one click on "Continue Task". Prompts 1 to 4 held when checked by hand. Prompt 5's reviews passed their gate but were thin and written by hand. | The review of a change is now written by the skill like every other report: both versions in numbers, the change line by line, every difference to judge, the claims of the earlier reports checked. Names the code reads before it sets them are no longer renamed. |
| Dry run | All five prompts with an agent in Bob's place, on a clean copy: all ten gates passed with no fault in the skill. | The review now tells a rename from a correction, and the list of commented-out code is more complete. |
| Clean run 3 | All ten gates passed at the first attempt, 54 Bob coins, one click on "Continue Task"; prompts 2 to 5 took one hour. Checked by hand, the code held: every correction, removal, path change and rename was right, with one exception: the clean-up removed four comments that explain live code (the events are unchanged). Bob's own words were wrong in places: a correction tagged with the wrong finding, and in the final reviews a wrong reason, a miscounted list and a question that calls a proposed change done. | The clean-up refuses to remove a comment that stands right above a live line that stays. The review lists each removed comment that is not code on its own, with the line it explained, and no longer counts a merge of two names, or a name that only gained its prefix, as a rename. A correction that names the wrong finding is refused. |
| Clean run 4 | All ten gates passed, about 42 Bob coins, prompts naming the exact files. Checked by hand: every correction, removal, rename and path change right; the comments that explain live code kept; the reviews list the corrections apart from the renames; Bob held back a change of event flow as a question for the client. | Three gaps closed, each generic: a command gets the file the request names, not its folder; switching commented-out code back on is a question for the owner; after a long chat is summarized, Bob checks every report the request asks for, not only the tasks it remembers. |
| Clean run 5, untouched | All ten gates passed with no intervention: nothing typed but the five prompts, no stop and no re-send, 44 minutes and about 53 Bob coins. Every step checked by hand was right; the final reviews found no unintended change and raised the client's open decisions as questions. | Nothing. |
| Clean runs with the 41 kinds | All ten gates passed, and a separate script checked every copy on its own: the fix adds no line, the clean-up removes only comments, the standardization changes names or path prefixes only, and both reviews find no unintended difference. The latest run: 14 minutes and about 25 Bob coins, with every prompt doing its own step for both inputs. | Gaps closed, each generic: a correction of Bob's that writes more lines than it replaces, or uses a ? operator new to the code, becomes a question for the owner; the clean-up removes the commented-out lines from the plan's own list, so Bob no longer reads the file again; after a long chat is summarized, the status names a step done for one input and not the other; a removed assignment that an included file may still read becomes a question for the owner. |
What the rehearsals teach
- Where the skill makes the change, the result is the same and right every time. Corrections that have one possible form, removal of the commented-out code the skill listed, renames from the skill's proposals, one rule for a path prefix.
- Where Bob writes new code itself, the result varies from run to run and was sometimes wrong while its gate passed: rewritten log lines, new lookup files, a split into modules. Those are kept out of the five prompts until they hold.
- A gate is necessary, not sufficient. It proves the work is complete and accounted for, not that it is right. Read each step's report and diff before sending the next prompt.
- Read what Bob writes in its own words. The skill checks every line of code and every count it produces; it cannot check a reason, a question or a number Bob types into a review. Those were wrong in places, so the reviews are read before they are presented.
- Leave the client's decisions to the client. When the review leaves a finding to the owner, the fix lists its correction as a proposal with a specific question instead of applying it.
- Long chats need a way back. Bob summarizes a long chat on the way; the
statuscommand lets it see from the files which tasks are done and which request it is answering. - Plan for the turn limit. Five prompts on two inputs take more than Bob's default 100 turns per task; raise the limit before the session or be ready to click "Continue Task".
Running it
- Open the workspace folder itself in IBM Bob, not its parent.
- In Bob's settings, raise the limit on the number of turns of a task above its default of 100; the five prompts in one chat need more.
- Start a new chat in the built-in Agent mode and send the five prompts in order, one after the other, and nothing else.
- After each prompt, Bob's commands should have printed
Gate: passedtwice, once for each input. Read the reports before sending the next prompt.