CPU and Memory Checks not working if windows ist installed with a non-english version

[K0nne]

Hello,

it came to our attention that the checks Invoke-IcingaCheckCPU and Invoke-IcingaCheckMemory were not working on a specific system. Upon investigating the issue we found, that the system was installed with a german edition of windows server 2016.

The problem seems to be, that even the performance counters are translated in german language:

Processor

icinga> Show-IcingaPerformanceCounterCategories -Filter '*Processor*'

icinga> Show-IcingaPerformanceCounterCategories -Filter '*Prozessor*'
Prozessor
Logischer Prozessor für Hyper-V-Hypervisor
Virtueller Prozessor des Hyper-V-Hypervisorstamms
Netzwerkschnittstellenkarten-Aktivität pro Prozessor
Netzwerkaktivitätszyklen pro Prozessor
Prozessorinformationen

icinga> Show-IcingaPerformanceCounters -Counter Prozessorinformationen
\Prozessorinformationen(*)\dpc-zeit (%)
\Prozessorinformationen(*)\privilegierte zeit (%)
\Prozessorinformationen(*)\dpcs in warteschlange/s
\Prozessorinformationen(*)\dpc-rate
\Prozessorinformationen(*)\% prozessorleistung
\Prozessorinformationen(*)\leerlaufzeit (%)
\Prozessorinformationen(*)\prozessorstatusflags
\Prozessorinformationen(*)\% c3-zeit
\Prozessorinformationen(*)\c3-übergänge/s
\Prozessorinformationen(*)\prozessorzeit (%)
\Prozessorinformationen(*)\interruptzeit (%)
\Prozessorinformationen(*)\% der maximalen frequenz
\Prozessorinformationen(*)\parkstatus
\Prozessorinformationen(*)\interrupts/s
\Prozessorinformationen(*)\benutzerzeit (%)
\Prozessorinformationen(*)\% c1-zeit
\Prozessorinformationen(*)\zeitinterrupts/s
\Prozessorinformationen(*)\prozessorauslastung
\Prozessorinformationen(*)\privilegierte auslastung
\Prozessorinformationen(*)\c2-übergänge/s
\Prozessorinformationen(*)\kennzeichen für das leistungslimit
\Prozessorinformationen(*)\durchschnittliche leerlaufzeit
\Prozessorinformationen(*)\prozessorfrequenz
\Prozessorinformationen(*)\unterbrechungen des leerlaufs/s
\Prozessorinformationen(*)\% leistungslimit
\Prozessorinformationen(*)\% prioritätszeit
\Prozessorinformationen(*)\% c2-zeit
\Prozessorinformationen(*)\c1-übergänge/s 

Memory

icinga> Show-IcingaPerformanceCounterCategories -Filter '*Arbeitsspeicher*'
Arbeitsspeicher
Arbeitsspeicher für NUMA-Knoten
Hyper-V-Dienst für die Integration des dynamischen Arbeitsspeichers

icinga> Show-IcingaPerformanceCounterCategories -Filter '*Memory*'
.NET Memory Cache 4.0

icinga> Show-IcingaPerformanceCounters -Counter Arbeitsspeicher
\Arbeitsspeicher\seitenlesevorgänge/s
\Arbeitsspeicher\zugesicherte bytes
\Arbeitsspeicher\freie und nullseitenlisten - bytes
\Arbeitsspeicher\nicht-auslagerungsseiten (bytes)
\Arbeitsspeicher\standbycache- bytes mit normaler priorität
\Arbeitsspeicher\cachebytes (max.)

In this case, the following message is thrown:

[UNKNOWN]: Icinga Invalid Configuration Error was thrown: PerfCounterCategoryMissing: Category "xxx" not found   

The specified Performance Counter category was not found on this system. This could either be a configuration error on your local Windows machine or a wrong usage of the plugin. Please check on different Windows machines if this issue persis. In case it only occurs on certain machines it is likely that the counter is simply not present and the plugin can not be processed.

Btw, there's a typo in the error message --> "persis"

[LordHepipud]

Thank you for the issue. How exactly was this system installed? My test machines are all installed by using the German localization and I never run into this issue so far.

It seems like the english counter part is not existing at all on the system, which makes me wonder on how other tools are querying the data required for runtime.

What happens if you try to fetch a counter:

Show-IcingaPerformanceCounterInstances -Counter '\Processor(*)\% processor time';

[K0nne]

The system was installed with a windows server 2016 and an additional german language pack.
The command seems to work just fine. Not sure why the checks are failing. They are executed by agent.

icinga> Show-IcingaPerformanceCounterInstances -Counter '\Processor(*)\% processor time';

Name                           Value
----                           -----
_Total                         \Processor(_Total)\% processor time
0                              \Processor(0)\% processor time
1                              \Processor(1)\% processor time


icinga>

Other people also stumbled over this in the past:
https://stackoverflow.com/questions/5581859/retrieve-performance-counter-value-in-a-language-independent-way

[LordHepipud]

I think here is a different issue happening. The problem is that the counters can be fetched just fine, because the way we implemented this always uses the englisch names.

How ever, we also added a test to check if a counter or category is available before doing anything else in some cases and the base code for this is using the same handling as Show-IcingaPerformanceCounterCategories

So in general this should mean the checks itself are running fine, but the test if a counter category is present or not fails because of this.

Right now I have no easy solution to fix this, as I'm checking for the category first (Processor) and break if this is not present, which oddly can happen on certain systems.

This will require additional investigation and will not be fixed with v1.4.0, but possibly with a later bugfix release.

[LordHepipud]

There is a new PR available on the Framework which will change on how the test for Performance Counter categories is done.

Could you please test this?

Icinga/icinga-powershell-framework#248

[K0nne]

The PR works. Thank you very much!

[LordHepipud]

Resolved with Icinga/icinga-powershell-framework#248

[log1-c]

hello @LordHepipud :)

i'm not sure if I understand the fix correctly, but I don't see why it should solve the counter problem.
We currently have two identical Windows servers (Windows Terminal servers) installed with:

• Icinga2-v2.12.3-x86_64
• icinga-powershell-framework-1.4.1
• icinga-powershell-plugins-1.4.0
• icinga-service-v1.1.0
• Framework Code Cache enabled

Both are checked via the exe-files of the Agent normally, but for testing purposes we gave them both one cpu check via the powershell framework.

What was confusing is, that the check via powershell mostly worked on one of the systems, but not on the other.

Applying the fix did not solve the problem. The checks on one server would not work, the other did mostly.

Turns out that we had enabled the experimental API features on the mostly working machine before (for testing if this reduces the CPU load of the powershell framework checks).
Afteractivating the experimental features the checks started working.

^ not working until API feature enabled
v working (mostly)

Maybe you can make sense of why the API feature makes the performance counter "detectable"?!

[LordHepipud]

Hello,

it shouldn't make a difference if system is checked by the API or the local handling. All that matters is, that the fix moved away by checking if the counter is present on the registry, but validates it by creating an object of the Processor category for example.

With the fix, both checks should work as you are able to call Show-IcingaPerformanceCounterInstances. What happens if you run

Test-IcingaPerformanceCounterCategory 'Processor'

On both systems, it should return true.

If not, what happens if you run

New-Object System.Diagnostics.PerformanceCounterCategory('Processor');

on both systems.

[log1-c]

This is the return of both systems currently:

PS C:\Windows\system32> Test-IcingaPerformanceCounterCategory 'Processor'
False
PS C:\Windows\system32> New-Object System.Diagnostics.PerformanceCounterCategory('Processor')|fl


CategoryName : Processor
CategoryHelp : The Processor performance object consists of counters that measure aspects of processor activity. The
               processor is the part of the computer that performs arithmetic and logical computations, initiates
               operations on peripherals, and runs the threads of processes.  A computer can have multiple processors.
                The processor object represents each processor as an instance of the object.
CategoryType : MultiInstance
MachineName  : .

And for the sake of completeness, this same calls for the german notation^^

PS C:\Windows\system32> Test-IcingaPerformanceCounterCategory 'Prozessor'
True
PS C:\Windows\system32> New-Object System.Diagnostics.PerformanceCounterCategory('Prozessor')|fl


CategoryName : Prozessor
CategoryHelp : Das Prozessorleistungsobjekt setzt sich aus Leistungsindikatoren zusammen, mit denen Aspekte der
               Prozessoraktivität gemessen werden. Der Prozessor ist der Teil des Systems, von dem arithmetische und
               logische Berechnungen ausgeführt, Vorgänge auf Peripheriegeräten initialisiert und Programme im System
               ausgeführt werden. Ein Computer kann mehrere Prozessoren besitzen. Vom Prozessorobjekt wird jeder
               Prozessor als Instanz des Objekts repräsentiert.
CategoryType : MultiInstance
MachineName  : .

[LordHepipud]

Just to be sure - are you using the latest master of the Icinga PowerShell Framework and rebuild your Framework cache in case it is enabled with Write-IcingaFrameworkCodeCache.

This doesn't make any sense on why Test-IcingaPerformanceCounterCategory return false, while the object exists.

Also: Is there another Framework folder present within the modules directory? It could rarely happen, that an older version is loaded, even when the name ist set to icinga-powershell-framework.bak for example.

[log1-c]

sorry for the late reply.

The systems have v1.4.1 of the framework installed and I added the Test-IcingaPerformanceCounterCategory fix manually.
I'm not sure if if I explicitly ran the Write-IcingaFrameworkCodeCache afterwards, I assume I didn't^^

I just did the same procedure again, as the server got redeployed yesterday.
Manually edited the Test-IcingaPerformanceCounterCategory and ran the Write-IcingaFrameworkCodeCache afterwards.

PS C:\Windows\system32> Write-IcingaFrameworkCodeCache
# in a new powershell
PS C:\Windows\system32> use-icinga
PS C:\Windows\system32> Test-IcingaPerformanceCounterCategory 'Processor'
True

I also have the second machine with the same setup but here I disabled the API features.

so it looks like the fix is working :)
Though I will monitor the state changes now and check if they still occur, though I guess they won't :)

older framwork versions are not/ were never present, as those VMs get redeloyed every couple of days from scratch.

2h later: no state changes yet, so that looks good.
With the "old" test there were about 1-2 state changes per hour, as seen in the screenshots posted previously