iSystems protects their intellectual property by tying their software to
an underlying physical computer with the use of a "computer modifier";
this 8-character code is used by their licensing system to keep things
from running where they don't belong.
When the modifier changes unexpectedly, some or all of Evolution will stop
working, requiring a new license key from iSystems support, and this has
been known to happen at highly inconvenient times.
While researching a customer who had sporadic and unexplained
de-licensing issues, I was able to determine what goes into the
modifier in order to see what changed between any two points in time.
This paper summarizes that research.
What's in the modifier?
The modifier is an eight-character hexadecimal number (32 bits) that
provides a "fingerprint" of the underlying hardware, and Evo's registration
key is tied to it. This helps forestall theft of iSystems' software.
The modifier can be seen in the Evo Management Console after navigating
to Configuration » License:
These items go into the modifier:
-
The registry key HLKM\Software\Microsoft\Windows NT\CurrentVersion\
contains to two strings values, RegisteredOwner and RegisteredOrganization,
which both feed into the modifier. These are often set at OS-install time, and in older operating
systems (XP, Server 2003) it shows up on the Properties page of My Computer, though
it's a bit less obvious in newer operating systems:
Do not change these casually!
-
The volume serial number of all fixed hard drives, but ignoring removable
media. External USB drives are sometimes fixed and sometimes removable,
so unless you know that it's removable, assume that adding a USB
hard drive will change the modifier. USB flash/thumb drives are almost
always disregarded as removable.
-
The system Processor Type and OEM ID; these are typically set at
installation time and are not typically changed (intentionally or
inadvertently) by the user. It's not clear what purpose these
values serve in Windows. The values for these fields is taken
fromn the GetSystemInfo() Win32 API call.
-
The address of the Ethernet card. This is determined indirectly,
so it's not entirely clear how multi-NIC configurations are
resolved into a single MAC address.
The evo-modinfo tool
In the cource of this research, I created a tool, evo-modinfo.exe,
that reports the items that go into the modifier (though it does not
compute the modifier itself). This program runs in a command window
and reports this way:
evo-modinfo 1.0.5 - 2008-03-19 - http://www.unixwiz.net/evo/
Report for computer "EVO2" run on: Mon Aug 20 20:26:36 2012
ITEMS MAKING UP MODIFIER:
Reg Owner "Steve Friedl"
Reg Organization "StevePay, LLC"
OEM ID 0x00000000
Proc Type 0x0000024a
MAC Address 00-0C-29-D5-17-FF
Fixed drives:
C:\ serial 0x18D1F414
A typical practice is to save the output to a file (say, c:\bin\evo-modinfo.txt)
to provide a baseline for comparison when asking "ok, what changed?". I make it a
practice to install this on all customer middle tier machines in the C:\BIN directory.
This tool may be downloade from here.
Fixing broken modifiers
The modifier is used, of course, directly for licensing of the Request Broker,
and if it changes, you must request a new license key from support, but it's
also used in additional places in more subtle ways. Mainly, it's used in the
password-encryption algorithm for middle-tier configuration files.
This section lists the areas where the modifier is used, what it looks
like when it's broken, and how to fix it. But note that if a modifier
changes on a machine, it breaks all the parts that use it (it generally
won't be mix and match).
It's important to note that each machine has its own modifier, and not all
Evolution services are affected. This table shows which Evo services
use the modifier in one way or another.
Evolution service |
Modifier impact? |
Evo Management Console | YES |
Request Broker | YES |
ADR Client | YES |
ADR Server | YES |
Evo Deployment Manager | no |
Request Processor | no |
Remote Relay | no |
API Adapter | no |
If a machine hosts only unaffected services, a modifier change won't
have any impact (and probably won't even be noticed).
In this section, we'll use the hypothetical environment variable %EVO% to refer
to where the Evolution services (but not the client!)
are installed on that system. This is usually one of:
C:\Program Files\Evolution or C:\Program Files (x86)\Evolution
or perhaps installed on a different drive, such as D:\. This variable
doesn't actually exist on the machine.
Generally speaking, when a modifier changes, the first thing you have to
fix is the Evo MC login user, because all the other changes are done with that
tool.
- Passwords for EMC login users
- The password for all Evo Management Console users are encrypted with
the modifier, and when that changes, all the passwords are effectively invalidated,
preventing you from logging in at all. You have to fix this before you can do
anything else.
-
To verify and fix:
- Using the browser, attempt to login to the Evo MC
- If the usual credentials don't work, try user=admin and password=evolution
- If none of them work, the modifier has probably changed.
- To fix, in the Windows "Services" control-panel applet,
stop the Evolution Management Console service
- Using Notepad, edit the file %EVO%\ManagementConsole\EvMgmtCon.cfg. It will
usually look something like this:
%EVO%\ManagementConsole\EvMgmtCon.cfg
[UserAccount]
admin=0100FFF4F03A157CCC80,3 «— delete this line
[DeploymentManager]
EVO1=
EVO2=
...
- Save the file
- In the Windows Services control-panel applet, start the Evo MC service again
- In your browser, login to the Evo Management Console with user=admin
and password=evolution
- If desired, navigate to Configuration » MC Users
and update the user accounts with new passwords
- Encrypted Firebird passwords
- When Evolution accesses the client files on the Linux database server, it does
so via the Firebird user EUSER, and that account has a password. If the
password is scrambled due to a changed modifier, Evo won't be able to access
any database files.
-
To verify and fix:
- Using the browser, Login to the Evo MC
- Make sure the Request Broker is running
- Nav to Configuration » Database Access:
- If the "EUSER password" field is blank:
- Re-enter the password (it's usually pps97)
- Click [Save Changes]
-
Note that the Firebird EUSER password is not something you ever use from
within Evolution itself - it's strictly a server configuration matter.
- Superuser account name
-
When Evolution needs to run key processes, such as an actual payroll, it switches internally
to a "system user" that has rights to do things that the underlying user may not. The name
of this user is encrypted, and if incorrect, generates a "Failed to switch to SU mode".
-
To verify and fix:
- Using the browser, login to the Evo MC
- Make sure the Request Broker is running
- Nav to Configuration » System User:
- If the User field is blank:
- Enter the superuser account name (usually super)
(there is no password associated with this account)
- Click [Save Changes]
-
Note: The name of the account is not at all security sensitive, so I've filed a
feature request with iSystems to have them remove the encryption from this field entirely.
- E-Mail configuration
- When Evo needs to send non-VMR emails (such as errors from the the queue),
it does so via the configuration in the Management Console, and if the SMTP account
users a password, it's encrypted with the modifier.
-
Note that many SBs have email servers that do not require authentication, so
the username and password fields are not used at all; this requires knowledge
of your local configuration.
-
To verify and fix:
- Using the browser, login to the Evo MC
- Make sure the Request Broker is running
- Nav to Configuration » EMail Notification:
- If the Username and Passwords are shown with *****, then it's probably OK
- If they are blank, either they're not required at all, or the modifier broke them.
To find out, click, [Send Test Email] and see whether the users mentioned in
the "Notifications are sent to" box receive the email.
- If they were not received, enter the username and password if you know it,
or see your local IT person for further troubleshooting.
-
Note that the configuration of VMR email is done from within the Evo client,
and that does not use the modifier, so won't need to be updated. If the Evo MC
email configuration is not working, it shouldn't imact Evo operations in a major way.
- Invalid RB License
- When the modifier changes, the Request Broker reports and invalid license (assuming
the EMail configuration is set properly), and it won't allow any Evo processing to run.
Evo is completely down in this case, requiring immediate action.
-
To verify and fix:
- Using the browser, login to the Evo MC
- Nav to Maintenance » Service Control
- Stop all Evo services
- Start the Request Broker
- Nav to Configuration » License
- If it shows License is invalid, a new license key must be requested.
- Compose an email to iSystems support, including in your message:
- Say explicitly that you're requesting a new license key
- Note whether it's a test or a production environment
- Include your serial number (from the MC License page). Production serials
are in the form 10XXX, while test serials are in the form 20XXX
- Include the "Computer modifier" shown at the top of the page, either
copying-and-pasting to your email, or typing it very carefully. If the
modifier is not correct, the license won't work.
- A mention of whether you're down now or not (a test
environment may not be time critical)
- If you're down, perhaps followup with a phone call to support
-
Support will respond with an email including your license key, which is quite long
and can only be reasonably cut and pasted into the Evo MC; it's best to perform
the next step while you have the email from support available.
-
To apply a new license key:
- Using the browser, login to the Evo MC
- Make sure the Request Broker is running
- Nav to Configuration » License
- Copy (using control-C) the license key to your clipboard and paste (using control-V) into the Registration code field
- Click [Save Changes]
- Observe that the license errors went away and that licensed products are shown:
- Stop, then start, all Evo services
- Return to the Configuration » License page to verify all is well
- Using the Evolution desktop client, attempt to login to Evo
- ADR client or server configuration
-
Most Evo service bureaus run some kind of replication using the Evolution Asynchronous Data Replication,
many to the iSystems hotsite. Some service bureaus also host their own ADR server.
-
All usernames and passwords used by ADR are encrypted with the modifier, and must be
updated if the modifier changes. Not that many service bureaus run the ADR client and
server on different machines from the
-
To fix the ADR client:
- Using the browser, login to the Evo MC
- Nav to Monitoring » AC Status
- Verify service status. If it has a green check:
... then all is well and no intervention is required.
-
If it shows a red X for the connection status:
... then the username and password for the ADR server must be re-entered.
- To fix, nav to Configuration » AC Config:
-
Enter the username and password only, then [Save Changes].
Do not change the other values from what's there now, as the
examples in this screenshot may not apply to your configuration.
-
Nav back to Monitoring » AC Status and verify that the
service has reconnected (it might take a minute or so to reconnect; click
the AC Status tab a few times to refresh).
Microsoft's KB2603229 update
A Windows Update published from Microsoft in late 2011 broke the modifier
on 64-bit Server 2008 R2, but fortunately it's easy to work around if the
workaround is performed before the update is applied.
On 64-bit operating systems, there's a separate (and parallel) section of the
registry that 32-bit programs use, and for whatever reason, the RegisteredOwner
and RegisteredOrganization fields in the 32-bit registry don't always match
the corresponding values in the 64-bit registry.
KB2603229 "fixes" this by
copying the values from the 64-bit registry tree down to the 32-bit tree, and
since Evolution is a 32-bit program, it sees this change and breaks the modifier.
It's hard to think of a Windows Update more narrowly tailored to breaking
the Evo modifier than KB2603229, and it's unclear why this was an issue important
enough to have an update in the first place.
In any case there are a number of approaches for dealing with it (all before
the fact).
- On a new server, apply this update prior to configuring and licensing
Evolution. This update has been out long enough that it ought to be applied
along with other pending Windows Updates, and once applied, it won't ever
be offered again even if the registry values in question don't match.
-
Prior to applying the update, use REGEDIT and manually edit the 64-bit versions
of the registry values to match the current 32-bit values, so that when
they're copied back, there won't be any visible change to 32-bit Evo. The registry
values are:
32bit: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows NT\currentversion\RegisteredOrganization
32bit: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows NT\currentversion\RegisteredOwner
64bit: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\currentversion\RegisteredOrganization
64bit: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\currentversion\RegisteredOwner
If you find that the two versions already match, the update won't have any impact (though you
should still apply it anyway just so it's out of the way).
-
Apply the update (which copies 64-bit » 32-bit trees), then manually edit
the 32-bit registry to put back the prior values. This may be the best option if
you have a copy of prior evo-modinfo output handy.
-
Mark the update as "Do not offer again"; this will prevent the update from being
applied at all. This is not recommended, because it will remain as a land mine
to be stepped on down the road: far better is to attend to it and be done with it.
-
Apply the update and plan on rebooting, then fully relicensing Evo. If you don't have
the original 32-bit values of the registry handy, this is probably your only option.
Even given these workarounds, it's wise to reboot the server after applying this
(and other) Windows Updates, then verify that Evo licensing hasn't changed.
Timing of modifier changes
A complicating factor is that though the factors that go into the modifier
are reflected immediately, the actual modifier is not recomputed until later,
when the services that rely on them start. This means that additional modifier
problems can lay dormant for some time even after they seem to have been fixed.
Scenario: if an unused network interface is disabled, it can impact
the factors computing the modifier, but it won't have any immediate
on Evolution.
But the next time the Evo services restart (perhaps while installing a
routine update), the Request Broker and ADR services will both recompute
the modifier and will require that they be relicensed and updated.
This could easily be a week after the change was made, making it
difficult to find (or remember) the root cause of the change.
But we're not finished: if only the regular Evo services were restarted
and fixed, the Evo Management Console has still been running from before
the change, and since it's not restarted, it's still using unchanged
passwords.
The next time the server reboots, the Evo MC restarts and finds the modifier
has changed, and none of the MC user credentials work any longer. This can
happen long after the original modifier problem was fixed, leading to
an puzzling concern over how the modifier changed "again".
The only way to be sure that whatever has changed in the modifier has
settled down is to reboot the server, which insures that all Evo services have
restarted, as well as any OS changes (NIC changes in particular) have for
sure been applied.
Best Practice:: If you discover that a modifier has changed, reboot
the server if you can before querying the modifier and getting a new license
key.
If rebooting at the moment is not practical due to pressing payroll work,
at least stop and start all Evo services before collecting the modifier and
relicensing. But: later, during the off hours, perform a full reboot
of the server and verify that all parts are still working.
Modifier Tips and Gotchas
There are a number of surprises when dealing with the modifier that we hope to
detail here.
-
Changing the hostname, domain, or IP parameters of a machine
do not impact the modifier or licensing.
-
The Linux DB server does not participate in the modifier in any way.
-
Virtualizing a machine with P-to-V (physical to virtual) almost always
changes the Ethernet address, which changes the modifier.
- Reformatting a hard drive almost always changes the volume
serial number, and though doing this with the C: drive means
reinstalling the operating system anyway, on secondary (D:, E:, etc.)
drives it's not out of the question that this could be done by adding
a new drive an extending a RAID array. This will change the modifier.
-
Disabling an inactive network card often changes the Ethernet address
that Evo uses for the modifier, breaking things. When configuring a
new middle-tier machine, I recommend disabling all non-active NICs early
in the process before any real Evo config has been done to avoid this.
First Published: 2013/04/17