Category Archives: Uncategorized

DPM 2016 – Reporting Services Server cannot connect to DPM Database

After installing System Center Data Protection Manager from scratch or after performing an upgrade from DPM 2012 R2, when you attempt to schedule a report to be mailed, you receive the following popup error or notice that when you generate a report you just receive a white page in your browser that continues to load indefinitely:


Reporting Services Server cannot connect to the DPM database.
To repair the configuration, follow steps for repairing DPM from DPM Setup Help.
ID: 3001
More Information
Just an endless loading page presented in your browser.

Unfortunately, following the repair steps suggested in the More Information link does not resolve the problem.

Resolution for SQL Server 2016 and later

  1. On the DPM server, open Computer Management, expand Local Users and Groups, click Groups, and create a new local group with the following information (replace items in red with the actual server name/hostname of your machine):
    1. Group name: DPMDBReaders$DPMSERVERNAME
    2. Description: This group is internally used by Microsoft System Center 2016 Data Protection Manager.
  2. On the DPM server, open Computer Management, expand Local Users and Groups, click Users, and create a new local user with the following information (replace items in red with the actual server name/hostname of your machine):
    1. User name: DPMR$DPMSERVERNAME
    2. Full name: DPMR$DPMSERVERNAME
    3. Description: This account is used for SQL reporting to generate reports for DPM 2016.
    4. Enter a strong password
    5. Check Password never expires
  3. Select your recently created DPMR$DPMSERVERNAME account and click on the Membership of tab
    1. Add the DPMDBReaders$DPMSERVERNAME group we created in step 1
    2. Click OK
  4. Start Microsoft SQL Server Management Studio and connect to the SQL instance used by DPM.
    1. Expand Security, right-click on the Logins, select New login
      1. Click on the Search… button and add the local group DPMDBReaders$DPMSERVERNAME
      2. Set the Default database to YourDPMDatabase

      3. Click on the User Mapping section, check the checkbox for YourDPMDatabase, and check the checkbox for the db_datareader role.
      4. Click OK
  5. In Microsoft SQL Server Management Studio, expand Databases, expand YourDPMDatabase, expand Security, expand Users, and right click Properties on the DPMDBReaders$DPMSERVERNAME group you granted access to in step 4.
    1. Click on Securables
    2. Click the Search… button
    3. Select Specific objects… and click OK
    4. Click the Object Types… button
    5. Check Stored precedures and click OK
    6. Click the Browse… button
    7. Check [dbo].[prc_MOM_Heartbeat_Get] and [dbo].[prc_MOM_ProductionServerGet] and click OK
    8. Click OK on the Select objects dialog box
    9. Place a checkbox in the Grant column for the Execute row.
      1. Make sure you do this step for both [dbo].[prc_MOM_Heartbeat_Get] and [dbo].[prc_MOM_ProductionServerGet], checking the box once will only update one of the storage procedures.
    10. Click OK
  6. Exit Microsoft SQL Server Management Studio.
  7. Open Reporting Services Configuration Manager and connect to the SqlServer and Instance hosting the DPM reports (as you go through this, replace the items in Red with your applicable values).
    1. Click on the Web Portal URL menu item and click on the listed URL for DPM.
    2. Click on the DPMReports_GUID folder to open the DPM reports page.
    3. Click on the DPMReporterDataSource Data Source to open its properties.
    4. Under Credentials, use the following configuration:
      1. Select the Using the following credentials radio button
      2. Type of credentials: Windows user name and password
      3. User name: DPMR$DPMSERVERNAME
      4. Password: EnterYourPasswordToTheAccount
      5. Ensure Log in using these credentials, but then try to impersonate the user viewing the report is unchecked
      6. Click the Test connection button
        1. Ensure it says Connected successfully

      7. Click Apply
    5. Close out of your web browser
  8. Back in the Reporting Services Configuration Manager window, complete the following steps
    1. Select the Service Account menu item
    2. Ensure Use built-in account is set to Network Service
      1. Check the Use built-in account radio button
      2. Set the account to Network Service
      3. Click Apply
      4. You will be prompted to save an encryption key. Save the key to location of your choosing, and type a password to be used to encrypt the file. Click OK
      5. You will then be prompted to specify an account with administrator privileges. Click OK to use your current account.
  9. Reboot the DPM Server

At this point, you should now be able to schedule e-mail reports without experiencing the original error and your reports should load properly!

DPM 2016 – Anonymous / Open Relay for SMTP Notifications

DPM 2016 is primarily geared towards using mail servers that require authentication (rightfully so, that’s a best security practice). However, many IT organizations have local mail relay servers with anonymous authentication that are used for several IT services in the organization. Unfortunately, DPM 2016 gets a bit wonky using unauthenticated mail servers and will likely give you a generic error that says:

Error ID: 2013
Details: The user name or password is incorrect

And if you ignore the error and head over to the notifications tab to configure a notification, you will be presented with another generic error:

An authentication error occured when trying to connect to the SMTP serve. (ID: 518)

You typed an incorrect user name, password, or SMTP server name. Type the correct user name or password to enable e-mail delivery of reports and alert notifications.

And if you are trying to configure scheduled emails you may receive an error about reporting services:

DPM Setup is unable to update the report server configuration to configure e-mail settings. (ID: 3040).

One thing I may do before getting too far ahead though is validate you can send an email from the DPM server. This can easily be done via PowerShell by executing the following command:

Send-MailMessage -SMTPServer localhost -To [email protected] -From [email protected] -Subject "Test Email from DPM Server" -Body "Howdy!  This is a test from the DPM Sever.  If you see this, mail relay is working!"

When executing the PowerShell command, it won’t return anything, but you should hopefully see a message in your mailbox. If you do, you’ve at least ruled out network/mail issues.

Once you’ve ruled out connectivity/the mail server, we will complete the following steps below to configure DPM.

  1. Configure E-mail for SQL Server Reporting Services
  2. Create a Local User Account
  3. Remove any artifacts left in the registry
  4. Update the SMTP settings in DPM.

Configuration

  1. Configure SQL Server Reporting Services
    1. Open Reporting Services Configuration Manager
    2. Sign into your DPM instance
    3. Select E-mail Settings and leverage the following configuration
      1. Sender Address: [email protected]
      2. SMTP Server: emailserver.yourdomain.com
      3. Authentication: No authentication

    4. Click Apply
  2. Create a local user account
    1. Open Computer Management, expand Local Users and Groups, select Users, and Create a new local user on the machine
      1. Create the user (I used anonemail as the account name, but anything can be specified)
      2. Remove all group membership
        1. This account doesn’t need to be a part of any group, including the Users group
        2. This account should not be a part of administrators (I’ve seen other blog posts mention you must use administrator, that is 100% not necessary and can be considered a security risk)
      3. Ensure the account is enabled
        1. A disabled account will not work
  3. Cleanup the registry
    1. Open registry editor (regedit.msc)
    2. Navigate to HKEY_LOCAL_MACHINE\Software\Microsoft\Microsoft Data Protection Manager\Notification
    3. Delete the following keys (if they exist):
      1. SmtpUserName
      2. SmtpPassword
  4. Reboot the DPM Server
    1. Technically, you could restart two services:
      SQL Server Reporting Services instance for DPM and the DPM service, but a reboot never hurts 😉
  5. Configure DPM to use SMTP relay
    1. Close out of the DPM and reopen
    2. Select Reporting, waiting for the screen to finish loading, and then select Action, Options
    3. Select the SMTP Server tab and enter
      1. SMTP sever name: relayserver.mydomain.com
      2. SMTP server port: 25
      3. “From” Address: [email protected]
      4. Username: .\localuserwecreatedearlier
        1. Ensure you have .\ to designate the user is local
      5. Password: LocalUserAccountPassword

    4. Click the Send Test E-Mail button and specify an email address to send a test email to validate all is well
    5. Success!
    6. Click OK on the Options window to save your settings

At this point, you should be able to relay emails through your open relay as well as schedule emails for reports without error.

DPM 2016 Installation – Error ID: 4387

When installing DPM 2016, you may get a really generic error during the “Prerequisites check” during installation. Looking online, there’s a ton of individuals that have this issue, but no one correlates the log files to specifically what is needed to solve each problem (yep, “each” problem, Error 4387 is a generic catch all for several issues during the prerequisits check).

Before I get into the article, the too long didn’t read (TLDR) version is make sure you are using both SQL Server 2016 (no service pack) and SSMS 16.5 or earlier to successfully install DPM 2016.

To get a bit more technical and find out what’s going on, open up the DPM Installation logs after you receive the error. The installation log files can be found by browsing to %ProgramFiles%\Microsoft System Center 2016\DPM\DPMLogs.  Documentation on where log files are stored by DPM can be found here: https://docs.microsoft.com/en-us/system-center/dpm/set-up-dpm-logging?view=sc-dpm-2016

Here’s a copy of my DpmSetup.log file, in which when looking through it, there isn’t a clear cut answer, just this generic line at the bottom ([3/1/2019 6:22:54 AM] *** Error : CurrentDomain_UnhandledException).

[3/1/2019 6:21:16 AM] Information : Microsoft System Center 2016 Data Protection Manager setup started.
[3/1/2019 6:21:16 AM] Data : Mode of setup = User interface
[3/1/2019 6:21:16 AM] Data : OSVersion = Microsoft Windows NT 10.0.14393.0
[3/1/2019 6:21:16 AM] Information : Check if the media is removable
[3/1/2019 6:21:16 AM] Data : Folder Path = C:\Program Files\Microsoft System Center 2016\DPM
[3/1/2019 6:21:16 AM] Data : Drive Name = C:\
[3/1/2019 6:21:16 AM] Data : Drive Type = 3
[3/1/2019 6:21:16 AM] Information : Check attributes of the directory
[3/1/2019 6:21:16 AM] Data : Folder Path = C:\Program Files\Microsoft System Center 2016\DPM
[3/1/2019 6:21:16 AM] Data : File Attributes = Directory
[3/1/2019 6:21:16 AM] Information : Check if the media is removable
[3/1/2019 6:21:16 AM] Data : Folder Path = C:\Program Files\Microsoft Data Protection Manager
[3/1/2019 6:21:16 AM] Data : Drive Name = C:\
[3/1/2019 6:21:16 AM] Data : Drive Type = 3
[3/1/2019 6:21:16 AM] Information : Check attributes of the directory
[3/1/2019 6:21:16 AM] Data : Folder Path = C:\Program Files\Microsoft Data Protection Manager
[3/1/2019 6:21:16 AM] * Exception : Ignoring the following exception intentionally => System.IO.FileNotFoundException: Could not find file 'C:\Program Files\Microsoft Data Protection Manager'.
File name: 'C:\Program Files\Microsoft Data Protection Manager'
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.File.GetAttributes(String path)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Wizard.InstallLocationValidation.CheckForDirectoryAttributes(String path)
[3/1/2019 6:21:16 AM] Information : Check if the media is removable
[3/1/2019 6:21:16 AM] Data : Folder Path = C:\Program Files\Microsoft System Center 2016\DPM\DPM\DPMDB
[3/1/2019 6:21:16 AM] Data : Drive Name = C:\
[3/1/2019 6:21:16 AM] Data : Drive Type = 3
[3/1/2019 6:21:16 AM] Information : Check attributes of the directory
[3/1/2019 6:21:16 AM] Data : Folder Path = C:\Program Files\Microsoft System Center 2016\DPM\DPM\DPMDB
[3/1/2019 6:21:16 AM] * Exception : Ignoring the following exception intentionally => System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\Program Files\Microsoft System Center 2016\DPM\DPM\DPMDB'.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.File.GetAttributes(String path)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Wizard.InstallLocationValidation.CheckForDirectoryAttributes(String path)
[3/1/2019 6:21:17 AM] Information : The setup wizard is initialized.
[3/1/2019 6:21:17 AM] Information : Starting the setup wizard.
[3/1/2019 6:21:17 AM] Information : <<< Dialog >>> Welcome Page : Entering
[3/1/2019 6:22:33 AM] Information : <<< Dialog >>> Welcome Page : Leaving
[3/1/2019 6:22:33 AM] Information : <<< Dialog >>> Inspect Page : Entering
[3/1/2019 6:22:41 AM] Information : Query WMI provider for path of configuration file for SQL Server 2008 Reporting Services.
[3/1/2019 6:22:41 AM] Information : Querying WMI Namespace: \\DPM-SERVER\root\Microsoft\SqlServer\ReportServer\RS_DPM\V13\admin for query: SELECT * FROM MSReportServer_ConfigurationSetting WHERE InstanceName='DPM'
[3/1/2019 6:22:42 AM] Data : Path of configuration file for SQL Server 2008 Reporting Services = C:\Program Files\Microsoft SQL Server\MSRS13.DPM\Reporting Services\ReportServer\RSReportServer.config
[3/1/2019 6:22:42 AM] * Exception :  => System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.SqlServer.Smo, Version=10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' or one of its dependencies. The system cannot find the file specified.
File name: 'Microsoft.SqlServer.Smo, Version=10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91'
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Helpers.MiscHelper.IsSqlClustered(String sqlMachineName, String sqlInstanceName)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Helpers.MiscHelper.IsMachineClustered(String sqlMachineName, String sqlInstanceName)

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

[3/1/2019 6:22:42 AM] * Exception :  => System.Management.ManagementException: Invalid namespace 
   at System.Management.ManagementException.ThrowWithExtendedInfo(ManagementStatus errorCode)
   at System.Management.ManagementScope.InitializeGuts(Object o)
   at System.Management.ManagementScope.Initialize()
   at System.Management.ManagementObjectSearcher.Initialize()
   at System.Management.ManagementObjectSearcher.Get()
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Helpers.WmiHelper.IsMachineClustered(String machineName, String instanceName)
[3/1/2019 6:22:42 AM] Information : OS >= win 8 , enable Dedupe role
[3/1/2019 6:22:53 AM] Information : output : True
.. 
 error : 
[3/1/2019 6:22:53 AM] Data : Path of inspection output xml = C:\Program Files\Microsoft System Center 2016\DPM\DPMLogs\InspectReport.xml
[3/1/2019 6:22:53 AM] Information : Instantiating inspect component.
[3/1/2019 6:22:53 AM] Data : Path of output xml = C:\Program Files\Microsoft System Center 2016\DPM\DPMLogs\InspectReport.xml
[3/1/2019 6:22:53 AM] Information : Deserializing the check XML from path : C:\Users\labuser.CONTOSO\AppData\Local\Temp\DPM8AFA.tmp\DPM2012\Setup\checks.xml
[3/1/2019 6:22:53 AM] Information : Loading the check XML from path : C:\Users\labuser.CONTOSO\AppData\Local\Temp\DPM8AFA.tmp\DPM2012\Setup\checks.xml
[3/1/2019 6:22:54 AM] Information : Deserialising the scenario XML from path : C:\Users\labuser.CONTOSO\AppData\Local\Temp\DPM8AFA.tmp\DPM2012\Setup\scenarios.xml
[3/1/2019 6:22:54 AM] Information : Loading the check XML from path : C:\Users\labuser.CONTOSO\AppData\Local\Temp\DPM8AFA.tmp\DPM2012\Setup\scenarios.xml
[3/1/2019 6:22:54 AM] Information : Getting scenarios for the product: DPM
[3/1/2019 6:22:54 AM] Information : Getting scenarios for DPM
[3/1/2019 6:22:54 AM] Information : Getting scenario for Mode:Install, DbLocation:Remote, SKU:Retail and CCMode:NotApplicable
[3/1/2019 6:22:54 AM] *** Error : Initialize the SQLSetUpHelper Object
[3/1/2019 6:22:54 AM] Information : [SQLSetupHelper.GetWMIReportingNamespace]. Reporting Namespace found. Reporting Namespace : V13
[3/1/2019 6:22:54 AM] Information : [SQLSetupHelper.GetWMISqlServerNamespace]. SQL Namespace found. SQL Namespace : \\DPM-SERVER\root\Microsoft\SqlServer\ComputerManagement13
[3/1/2019 6:22:54 AM] Information : Query WMI provider for SQL Server 2008.
[3/1/2019 6:22:54 AM] Information : Querying WMI Namespace: \\DPM-SERVER\root\Microsoft\SqlServer\ComputerManagement13 for query: Select * from SqlServiceAdvancedProperty where ServiceName='MSSQL$DPM' and PropertyName='Version'
[3/1/2019 6:22:54 AM] Information : SQL Server 2008 R2 SP2 instance DPM is present on this system.
[3/1/2019 6:22:54 AM] Information : Query WMI provider for SQL Server 2008.
[3/1/2019 6:22:54 AM] Information : Querying WMI Namespace: \\DPM-SERVER\root\Microsoft\SqlServer\ComputerManagement13 for query: Select * from SqlServiceAdvancedProperty where ServiceName='MSSQL$DPM' and PropertyName='Version'
[3/1/2019 6:22:54 AM] Information : [SQLSetupHelper.GetSQLDepedency]. Reporting Namespace and SQL namespace for installed SQL server which will be used as DPM DB. Reporting Namespace : \\DPM-SERVER\root\Microsoft\SqlServer\ReportServer\RS_DPM\V13\admin SQL Namespace : \\DPM-SERVER\root\Microsoft\SqlServer\ComputerManagement13
[3/1/2019 6:22:54 AM] Information : Check if SQL Server 2012 Service Pack 1 Tools is installed.
[3/1/2019 6:22:54 AM] Information : [SQLSetupHelper.GetSqlSetupRegKeyPath]. Registry Key path that contains SQL tools location: Software\Microsoft\Microsoft SQL Server\140\Tools\Setup\
[3/1/2019 6:22:54 AM] Information : Inspect.CheckSqlServerTools : MsiQueryProductState returned : INSTALLSTATE_DEFAULT
[3/1/2019 6:22:54 AM] *** Error : CurrentDomain_UnhandledException

Digging some more, I found that DPM seems to also place logs within the %temp% folder. Within this folder, I found that a tmpXXX.xml file was being created each time I ran through the installer and triggered an error. Upon opening the file, I see the following:

<?xml version="1.0" encoding="utf-16"?>
<WatsonInfo xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <ExceptionList>
    <ExceptionEntry>
      <Exception_x0020_Type_x000D__x000A_>System.ArgumentNullException</Exception_x0020_Type_x000D__x000A_>
      <StackTrace_x000D__x000A_>   at System.Version.Parse(String input)
   at System.Version..ctor(String version)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Inspect.InspectPrerequisites.CheckSqlServerTools(InspectContext context)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Inspect.Inspect.InitializeContext(String sqlMachineName, String sqlInstanceName, String reportingMachineName, String reportingInstanceName, ConnectionOptions wmiSqlConnectionOptions, ConnectionOptions wmiReportingConnectionOptions, Boolean isRemoteDb, Boolean isSqlClustered, List`1 sqlClusterNodes, Boolean isRemoteReporting, String oldSqlMachineName, String oldSqlInstanceName, ProductNameEnum productName, InspectModeEnum inspectMode, Boolean remoteTriggerJob)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Inspect.Inspect..ctor(String reportFilePath, String sqlMachineName, String sqlInstanceName, String reportingMachineName, String reportingInstanceName, ConnectionOptions wmiSqlConnectionOptions, ConnectionOptions wmiReportingConnectionOptions, Boolean isRemoteDb, Boolean isSqlClustered, List`1 sqlClusterNodes, Boolean isRemoteReporting, String oldSqlMachineName, String oldSqlInstanceName, InspectModeEnum inspectMode, InspectSkuEnum inspectSku, ProductNameEnum productName, InspectCCModeEnum ccMode, Boolean remoteTriggerJob)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Wizard.BackEnd.InstantiateInspect(String inspectFile, String sqlMachineName, String sqlInstanceName, String reportingMachineName, String reportingInstanceName, ConnectionOptions wmiSqlConnectionOptions, ConnectionOptions wmiReportingConnectionOptions)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Wizard.InspectPage.RunInspect()
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Wizard.InspectPage.InspectThreadEntry()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()</StackTrace_x000D__x000A_>
      <message>Value cannot be null.
Parameter name: input</message>
      <targetSite>System.Version Parse(System.String)</targetSite>
    </ExceptionEntry>
  </ExceptionList>
  <UICulture>en-US</UICulture>
  <Culture>en-US</Culture>
  <CLRVersion>4.0.30319.42000</CLRVersion>
  <OSVersion>Microsoft Windows NT 10.0.14393.0</OSVersion>
  <BucketingParametersValue>
    <Other>BC81BBF7</Other>
    <ExceptionPoint>System.Version.Parse</ExceptionPoint>
    <ExceptionName>System.ArgumentNullException</ExceptionName>
    <ModuleVersion>5.0.158.0</ModuleVersion>
    <ModuleName>SetupDpm.exe</ModuleName>
    <ApplicationVersion>5.0.158.0</ApplicationVersion>
    <ApplicationName>SetupDpm</ApplicationName>
  </BucketingParametersValue>
  <Info>Microsoft Data Protection Manager Exception Record</Info>
</WatsonInfo>

Looking through the above stack trace, I see hints that this is to SQL Server and in this case I’m receiving a null value for what looks like a version. So after reading other posts online, everyone said to downgrade to SQL Server 2016 RTM.

SQL Server 2016 version numbers: https://support.microsoft.com/en-us/help/3177312/sql-server-2016-build-versions

After downgrading to SQL Server 2016 RTM, I noticed I still received Error ID: 4387. This time I don’t see any files within the %temp% directory, but I did find in the DPMSetup.log file (within the DPMLogs directory) the following log:

[3/8/2019 5:13:09 AM] Information : Microsoft System Center 2016 Data Protection Manager setup started.
[3/8/2019 5:13:09 AM] Data : Mode of setup = User interface
[3/8/2019 5:13:09 AM] Data : OSVersion = Microsoft Windows NT 10.0.14393.0
[3/8/2019 5:13:09 AM] Information : Check if the media is removable
[3/8/2019 5:13:09 AM] Data : Folder Path = C:\Program Files\Microsoft System Center 2016\DPM
[3/8/2019 5:13:09 AM] Data : Drive Name = C:\
[3/8/2019 5:13:09 AM] Data : Drive Type = 3
[3/8/2019 5:13:09 AM] Information : Check attributes of the directory
[3/8/2019 5:13:09 AM] Data : Folder Path = C:\Program Files\Microsoft System Center 2016\DPM
[3/8/2019 5:13:09 AM] Data : File Attributes = Directory
[3/8/2019 5:13:09 AM] Information : Check if the media is removable
[3/8/2019 5:13:09 AM] Data : Folder Path = C:\Program Files\Microsoft Data Protection Manager
[3/8/2019 5:13:09 AM] Data : Drive Name = C:\
[3/8/2019 5:13:09 AM] Data : Drive Type = 3
[3/8/2019 5:13:09 AM] Information : Check attributes of the directory
[3/8/2019 5:13:09 AM] Data : Folder Path = C:\Program Files\Microsoft Data Protection Manager
[3/8/2019 5:13:09 AM] * Exception : Ignoring the following exception intentionally => System.IO.FileNotFoundException: Could not find file 'C:\Program Files\Microsoft Data Protection Manager'.
File name: 'C:\Program Files\Microsoft Data Protection Manager'
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.File.GetAttributes(String path)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Wizard.InstallLocationValidation.CheckForDirectoryAttributes(String path)
[3/8/2019 5:13:09 AM] Information : Check if the media is removable
[3/8/2019 5:13:09 AM] Data : Folder Path = C:\Program Files\Microsoft System Center 2016\DPM\DPM\DPMDB
[3/8/2019 5:13:09 AM] Data : Drive Name = C:\
[3/8/2019 5:13:09 AM] Data : Drive Type = 3
[3/8/2019 5:13:09 AM] Information : Check attributes of the directory
[3/8/2019 5:13:09 AM] Data : Folder Path = C:\Program Files\Microsoft System Center 2016\DPM\DPM\DPMDB
[3/8/2019 5:13:09 AM] * Exception : Ignoring the following exception intentionally => System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\Program Files\Microsoft System Center 2016\DPM\DPM\DPMDB'.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.File.GetAttributes(String path)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Wizard.InstallLocationValidation.CheckForDirectoryAttributes(String path)
[3/8/2019 5:13:10 AM] Information : The setup wizard is initialized.
[3/8/2019 5:13:10 AM] Information : Starting the setup wizard.
[3/8/2019 5:13:10 AM] Information : <<< Dialog >>> Welcome Page : Entering
[3/8/2019 5:13:55 AM] Information : <<< Dialog >>> Welcome Page : Leaving
[3/8/2019 5:13:55 AM] Information : <<< Dialog >>> Inspect Page : Entering
[3/8/2019 5:14:06 AM] Information : Query WMI provider for path of configuration file for SQL Server 2008 Reporting Services.
[3/8/2019 5:14:06 AM] Information : Querying WMI Namespace: \\DPM-SERVER\root\Microsoft\SqlServer\ReportServer\RS_DPM\V13\admin for query: SELECT * FROM MSReportServer_ConfigurationSetting WHERE InstanceName='DPM'
[3/8/2019 5:14:06 AM] Data : Path of configuration file for SQL Server 2008 Reporting Services = C:\Program Files\Microsoft SQL Server\MSRS13.DPM\Reporting Services\ReportServer\RSReportServer.config
[3/8/2019 5:14:06 AM] * Exception :  => System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.SqlServer.Smo, Version=10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' or one of its dependencies. The system cannot find the file specified.
File name: 'Microsoft.SqlServer.Smo, Version=10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91'
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Helpers.MiscHelper.IsSqlClustered(String sqlMachineName, String sqlInstanceName)
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Helpers.MiscHelper.IsMachineClustered(String sqlMachineName, String sqlInstanceName)

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

[3/8/2019 5:14:06 AM] * Exception :  => System.Management.ManagementException: Invalid namespace 
   at System.Management.ManagementException.ThrowWithExtendedInfo(ManagementStatus errorCode)
   at System.Management.ManagementScope.InitializeGuts(Object o)
   at System.Management.ManagementScope.Initialize()
   at System.Management.ManagementObjectSearcher.Initialize()
   at System.Management.ManagementObjectSearcher.Get()
   at Microsoft.Internal.EnterpriseStorage.Dls.Setup.Helpers.WmiHelper.IsMachineClustered(String machineName, String instanceName)
[3/8/2019 5:14:06 AM] Information : OS >= win 8 , enable Dedupe role
[3/8/2019 5:14:07 AM] Information : output : True
.. 
 error : 
[3/8/2019 5:14:08 AM] Data : Path of inspection output xml = C:\Program Files\Microsoft System Center 2016\DPM\DPMLogs\InspectReport.xml
[3/8/2019 5:14:08 AM] Information : Instantiating inspect component.
[3/8/2019 5:14:08 AM] Data : Path of output xml = C:\Program Files\Microsoft System Center 2016\DPM\DPMLogs\InspectReport.xml
[3/8/2019 5:14:08 AM] Information : Deserializing the check XML from path : C:\Users\labuser.CONTOSO\AppData\Local\Temp\DPM6EC.tmp\DPM2012\Setup\checks.xml
[3/8/2019 5:14:08 AM] Information : Loading the check XML from path : C:\Users\labuser.CONTOSO\AppData\Local\Temp\DPM6EC.tmp\DPM2012\Setup\checks.xml
[3/8/2019 5:14:08 AM] Information : Deserialising the scenario XML from path : C:\Users\labuser.CONTOSO\AppData\Local\Temp\DPM6EC.tmp\DPM2012\Setup\scenarios.xml
[3/8/2019 5:14:08 AM] Information : Loading the check XML from path : C:\Users\labuser.CONTOSO\AppData\Local\Temp\DPM6EC.tmp\DPM2012\Setup\scenarios.xml
[3/8/2019 5:14:08 AM] Information : Getting scenarios for the product: DPM
[3/8/2019 5:14:08 AM] Information : Getting scenarios for DPM
[3/8/2019 5:14:08 AM] Information : Getting scenario for Mode:Install, DbLocation:Remote, SKU:Retail and CCMode:NotApplicable
[3/8/2019 5:14:08 AM] *** Error : Initialize the SQLSetUpHelper Object
[3/8/2019 5:14:08 AM] Information : [SQLSetupHelper.GetWMIReportingNamespace]. Reporting Namespace found. Reporting Namespace : V13
[3/8/2019 5:14:08 AM] Information : [SQLSetupHelper.GetWMISqlServerNamespace]. SQL Namespace found. SQL Namespace : \\DPM-SERVER\root\Microsoft\SqlServer\ComputerManagement13
[3/8/2019 5:14:08 AM] Information : Query WMI provider for SQL Server 2008.
[3/8/2019 5:14:08 AM] Information : Querying WMI Namespace: \\DPM-SERVER\root\Microsoft\SqlServer\ComputerManagement13 for query: Select * from SqlServiceAdvancedProperty where ServiceName='MSSQL$DPM' and PropertyName='Version'
[3/8/2019 5:14:08 AM] Information : SQL Server 2008 R2 SP2 instance DPM is present on this system.
[3/8/2019 5:14:08 AM] Information : Query WMI provider for SQL Server 2008.
[3/8/2019 5:14:08 AM] Information : Querying WMI Namespace: \\DPM-SERVER\root\Microsoft\SqlServer\ComputerManagement13 for query: Select * from SqlServiceAdvancedProperty where ServiceName='MSSQL$DPM' and PropertyName='Version'
[3/8/2019 5:14:08 AM] Information : [SQLSetupHelper.GetSQLDepedency]. Reporting Namespace and SQL namespace for installed SQL server which will be used as DPM DB. Reporting Namespace : \\DPM-SERVER\root\Microsoft\SqlServer\ReportServer\RS_DPM\V13\admin SQL Namespace : \\DPM-SERVER\root\Microsoft\SqlServer\ComputerManagement13
[3/8/2019 5:14:08 AM] Information : Check if SQL Server 2012 Service Pack 1 Tools is installed.
[3/8/2019 5:14:08 AM] Information : [SQLSetupHelper.GetSqlSetupRegKeyPath]. Registry Key path that contains SQL tools location: Software\Microsoft\Microsoft SQL Server\140\Tools\Setup\
[3/8/2019 5:14:08 AM] Information : Inspect.CheckSqlServerTools : MsiQueryProductState returned : INSTALLSTATE_DEFAULT
[3/8/2019 5:14:08 AM] *** Error : CurrentDomain_UnhandledException

Looking at the above log, the last line hints we are looking for SQL Server Tools (in this case, what looks like some crazy old hints to depencies on SQL Server 2012). Unfortunately, installation of SQL Server 2016 will provide you the recommendation to grab SQL Server Management Studio 17.X, however DPM 2016 will only install with SQL Server Management Studio 16.5.X. You will need to uninstall the 17.X version of SSMS and install the 16.5.X build from the link below:
https://docs.microsoft.com/en-us/sql/ssms/sql-server-management-studio-changelog-ssms?view=sql-server-2017#download-ssms-1653

Alas! Upon installation and run through the DPM installation, no more Error 4387! Once DPM is installed, you can safely upgrade your SQL Server instance to 2017 if needed.

Hope this helps someone else! DPM can be picky and unforgiving in nature, but if you abide by exactly what their documentation calls out to a T and not venture anything outside of those parameters, you should be golden 🙂

https://docs.microsoft.com/en-us/system-center/dpm/install-dpm?view=sc-dpm-2016#setup-prerequisites

Closing notes, if the above items didn’t solve your problem. Please post your logs and let’s troubleshoot to document all solutions needed for all error logs. Thank you!

Installing Python Wheel files on an Azure App Service

Per Microsoft: Some packages may not install using pip when run on Azure. It may simply be that the package is not available on the Python Package Index. It could be that a compiler is required (a compiler is not available on the machine running the web app in Azure App Service).

Example, you may receive an error like this when trying to install a specific package (in this case, trying to install Pandas):

Command: "D:\home\site\deployments\tools\deploy.cmd"
Handling python deployment.
KuduSync.NET from: 'D:\home\site\repository' to: 'D:\home\site\wwwroot'
Copying file: 'requirements.txt'
Detected requirements.txt.  You can skip Python specific steps with a .skipPythonDeployment file.
Detecting Python runtime from runtime.txt
Detected python-2.7

Found compatible virtual environment.
Pip install requirements.
Downloading/unpacking Flask==0.12.1 (from -r requirements.txt (line 1))
Downloading/unpacking numpy==1.15.0rc2 (from -r requirements.txt (line 2))
Downloading/unpacking pandas==0.22.0 (from -r requirements.txt (line 3))
  Running setup.py (path:D:\home\site\wwwroot\env\build\pandas\setup.py) egg_info for package pandas

    Could not locate executable g77
    Could not locate executable f77
    Could not locate executable ifort
    Could not locate executable ifl
    Could not locate executable f90
    Could not locate executable efl
    Could not locate executable gfortran
    Could not locate executable f95
    Could not locate executable g95
    Could not locate executable effort
    Could not locate executable efc
    don't know how to compile Fortran code on platform 'nt'
    non-existing path in 'numpy\\distutils': 'site.cfg'
    Running from numpy source directory.

    d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\setup.py:385: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
      run_build = parse_setuppy_commands()
    D:\python27\Lib\distutils\dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
      warnings.warn(msg)
    d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
        Atlas (http://math-atlas.sourceforge.net/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [atlas]) or by setting
        the ATLAS environment variable.
      self.calc_info()

    d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
        Blas (http://www.netlib.org/blas/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [blas]) or by setting
        the BLAS environment variable.
      self.calc_info()

    d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
        Blas (http://www.netlib.org/blas/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [blas_src]) or by setting
        the BLAS_SRC environment variable.
      self.calc_info()

    d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
        Lapack (http://www.netlib.org/lapack/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [lapack]) or by setting
        the LAPACK environment variable.
      self.calc_info()

    d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
        Lapack (http://www.netlib.org/lapack/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [lapack_src]) or by setting
        the LAPACK_SRC environment variable.
      self.calc_info()

    D:\python27\Lib\distutils\dist.py:267: UserWarning: Unknown distribution option: 'define_macros'
      warnings.warn(msg)
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "D:\home\site\wwwroot\env\build\pandas\setup.py", line 743, in <module>
        **setuptools_kwargs)
      File "D:\python27\Lib\distutils\core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\dist.py", line 262, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\dist.py", line 287, in fetch_build_eggs
        replace_conflicting=True,
      File "D:\home\site\wwwroot\env\lib\site-packages\pkg_resources.py", line 614, in resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "D:\home\site\wwwroot\env\lib\site-packages\pkg_resources.py", line 857, in best_match
        return self.obtain(req, installer)
      File "D:\home\site\wwwroot\env\lib\site-packages\pkg_resources.py", line 869, in obtain
        return installer(requirement)
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\dist.py", line 338, in fetch_build_egg
        return cmd.easy_install(req)
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 613, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 643, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 833, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 1055, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 1043, in run_setup
        raise DistutilsError("Setup script exited with %s" % (v.args[0],))
    distutils.errors.DistutilsError: Setup script exited with error: Microsoft Visual C++ 9.0 is required (Unable to find vcvarsall.bat). Get it from http://aka.ms/vcpython27
    Complete output from command python setup.py egg_info:

Could not locate executable g77
Could not locate executable f77
Could not locate executable ifort
Could not locate executable ifl
Could not locate executable f90
Could not locate executable efl
Could not locate executable gfortran
Could not locate executable f95
Could not locate executable g95
Could not locate executable effort
Could not locate executable efc

don't know how to compile Fortran code on platform 'nt'
non-existing path in 'numpy\\distutils': 'site.cfg'
Running from numpy source directory.
d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\setup.py:385: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
  run_build = parse_setuppy_commands()
D:\python27\Lib\distutils\dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
  warnings.warn(msg)

d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  self.calc_info()

d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
    Blas (http://www.netlib.org/blas/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [blas]) or by setting
    the BLAS environment variable.
  self.calc_info()

d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
    Blas (http://www.netlib.org/blas/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [blas_src]) or by setting
    the BLAS_SRC environment variable.
  self.calc_info()

d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
    Lapack (http://www.netlib.org/lapack/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [lapack]) or by setting
    the LAPACK environment variable.
  self.calc_info()

d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
    Lapack (http://www.netlib.org/lapack/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [lapack_src]) or by setting
    the LAPACK_SRC environment variable.
  self.calc_info()

D:\python27\Lib\distutils\dist.py:267: UserWarning: Unknown distribution option: 'define_macros'
  warnings.warn(msg)

Traceback (most recent call last):
  File "<string>", line 17, in <module>
  File "D:\home\site\wwwroot\env\build\pandas\setup.py", line 743, in <module>
    **setuptools_kwargs)
  File "D:\python27\Lib\distutils\core.py", line 111, in setup
    _setup_distribution = dist = klass(attrs)
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\dist.py", line 262, in __init__
    self.fetch_build_eggs(attrs['setup_requires'])
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\dist.py", line 287, in fetch_build_eggs
    replace_conflicting=True,
  File "D:\home\site\wwwroot\env\lib\site-packages\pkg_resources.py", line 614, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "D:\home\site\wwwroot\env\lib\site-packages\pkg_resources.py", line 857, in best_match
    return self.obtain(req, installer)
  File "D:\home\site\wwwroot\env\lib\site-packages\pkg_resources.py", line 869, in obtain
    return installer(requirement)
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\dist.py", line 338, in fetch_build_egg
    return cmd.easy_install(req)
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 613, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 643, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 833, in install_eggs
   return self.build_and_install(setup_script, setup_base)
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 1055, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 1043, in run_setup
    raise DistutilsError("Setup script exited with %s" % (v.args[0],))

distutils.errors.DistutilsError: Setup script exited with error: Microsoft Visual C++ 9.0 is required (Unable to find vcvarsall.bat). Get it from http://aka.ms/vcpython27

----------------------------------------

Cleaning up...

Command python setup.py egg_info failed with error code 1 in D:\home\site\wwwroot\env\build\pandas
Storing debug log for failure in D:\home\pip\pip.log

An error has occurred during web site deployment.
\r\nD:\Program Files (x86)\SiteExtensions\Kudu\75.10629.3460\bin\Scripts\starter.cmd "D:\home\site\deployments\tools\deploy.cmd"

This guide is a reflection on how to use Wheel files to install Modules that cannot natively be installed via pip due to a compiler missing in the Azure App Service:

Microsoft Official documentation can be found here: https://docs.microsoft.com/en-us/azure/app-service/web-sites-python-configure#troubleshooting—package-installation

Tutorial

  1. Modify requirements.txt file
    1. Add the following item as the first line to the document:
      1. –find-links wheelhouse
        1. Note: If you do not have a requirements.txt file, you can simply create a new text document and add this line to it.  The requirements.txt file is what allows the Azure App Service to automatically go out and try and download packages you may need for your application.  Official documentation on this file is found here: https://docs.microsoft.com/en-us/azure/app-service/web-sites-python-configure#package-management
    2. Navigate to the Kudu Debug Console by going to https://yourappservice.scm.azurewebsites.net/DebugConsole
    3. Within the debug console, navigate to your version of Python.
      1. Note: The default Python versions in an Azure App Service are 2.7 and 3.4; however since Wheel will need to install some files, you cannot leverage the default directories of D:\Python27 for v2.7 and D:\Python34 for v3.4
      2. In this case, I’d recommend leveraging Extensions to install whatever version of Python.  Documentation on this can be found here: https://blogs.msdn.microsoft.com/pythonengineering/2016/08/04/upgrading-python-on-azure-app-service/
    4. Install the Python Wheel module:
      1. python.exe -m pip install wheel
    5. Obtain Wheel files
      1. Option 1: Build your own wheel files
        1. Execute the following command:
          1. python.exe -m pip wheel -r D:\home\site\wwwroot\requirements.txt -w wheelhouse

      2. Option 2: Obtain Wheel files
        1. Create a wheelhouse folder within your python directory
          1. mkdir wheelhouse

        2. Copy whl files to this directory
          1. You can obtain wheel files from PyPi or from Laboratory for Fluorescence Dynamics, University of California, Irvine.
            1. PyPi: Search for the module and then clicking on the Download Files button
              1. https://pypi.org/
            2. Laboratory for Fluorescence Dynamics, University of California, Irvine: Simply download the appropriate whl file listed on the page below
              1. https://www.lfd.uci.edu/~gohlke/pythonlibs/
    6. Install Modules
      1. Manual Install
        1. Execute the following command:
          python.exe -m pip install –upgrade -r D:\home\site\wwwroot\requirements.txt

      2. Deployment Install (from CI/CD pipeline)
        1. Configure .deployment and deploy.cmd file
          1. Official documentation on this can be found here: https://github.com/projectkudu/kudu/wiki/Custom-Deployment-Script
          2. .deployment file
            1. [config]
              command = deploy.cmd
          3. deploy.cmd file (modify the python directory to reflect your version)
            1. :: 1. Install Wheel
              echo Configure Wheel
              D:\home\python364x64\python.exe -m pip install wheel:: 2. Install packages
              echo Pip install requirements.
              D:\home\python364x64\python.exe -m pip install –upgrade -r D:\home\site\wwwroot\requirements.txt

At this point, the modules in question should be installed and ready for use! 🙂

Setting up WeeWX with a Raspberry PI

This is a quick setup guide on how to configure the open source software WeeWX for a Personal Weather Station (PWS).  I highly recommend you check out the WeeWX User Guide as this information is very well documented.  Here is a reflection of how I was able to get WeeWX installed on a Raspberry PI with a brand new weather station.

  1. Setup your Raspberry PI
    1. How to setup your Raspberry PI: http://jackstromberg.com/2018/03/setting-up-a-new-raspberry-pi-via-ssh/
      1. Note: Raspbian is a distribution based upon Debian.  In this case, we will follow the Debian instructions for setting up WeeWX.
        1. http://weewx.com/docs/debian.htm
  2. (Optional) Configure the Raspberry PI to be localized to your environment
    1. sudo raspi-config
      1. Here you can arrow down to Localization Options and configure the timezone to match that of your console/weather sensor.  Keeping time is critical, so if possible, try to keep the date/time between your weather station and the Raspberry PI as close as possible.
  3. Configure Apt-Get to look for the WeeWX packages
    wget -qO - http://weewx.com/keys.html | sudo apt-key add -
    sudo wget -qO - http://weewx.com/apt/weewx.list | sudo tee /etc/apt/sources.list.d/weewx.list
  4. Update your Raspberry-PI to use the latest packages
    sudo apt-get update
    sudo apt-get upgrade
  5. Before installation, ensure you have your console or device setup and connected to your Raspberry PI for WeeWX to pull the data
  6. Determine the interface the console is connected to (if using a directly attached data loggerm skip if using an IP based source)
    1. Execute the command dmesg and look for what interface the data logger is connected to
      1. In my example, you can see the data logger is connected to ttyUSB0
  7. Launch the installation wizard for weewx
    1. sudo apt-get install weewx
      1. Note: You will likely be prompted to install a few dependencies, type Y for yes to install them
  8. Installation
    1. Enter the location of your weather station: Santa’s Workshop, North Pole
    2. Enter in the latitude, longitude of your weather station
      1. Note: If you don’t have GPS, you can easily find this by using Bing Maps or Google Maps, navigating to your location, and right clicking.
        1. For Bing, it will just show you the lat/long values when you right click
        2. For Google, click on “What’s Here” and it will list these values
      2. Note: You can be more specific than 3 digits behind the decimal, so if you want to use a more specific set of coordinates like 40.689167, -74.044444, that is acceptable.
    3. Enter in your Altitude of where the weather station is
      1. You can use Google Earth to find the altitude or this tool here: https://www.freemaptools.com/elevation-finder.htm
    4. Set your preferred unit of measurement
      1. US (Imperial) or Metric
    5. Select your weather station type
      1. I.e. AcuRite, Vantage (if using Davis), etc.
    6. Select the interface the device is listening on
    7. For those using serial port, select the interface that the data logger is connected to.  You should have found this in step 4 above; if using ethernet, go ahead and type in the IP, Port, etc. of the data logger.
  9. At this point WeeWX is technically installed, however many individuals will want to present the WeeWX reports via webpage.  In this case, we’ll install nginx, which is a lightweight webserver
    1. sudo apt-get install nginx
      1. More details on this can be found here: http://www.weewx.com/docs/usersguide.htm#integrating_with_webserver
  10. Configure WeeWX to minimize disk IO
    1. Why do we need to do this?  Since Raspberry PI’s leverage SD cards, there is typically a finite number of reads/writes to the SD Card.  In this case, it is recommended to either leverage an external database/fileserver for WeeWX to write its reports.  Alternatively, we can also configure WeeWX to leverage ram to host the reports, which will prevent IO to the SD card (in this case, theoretically increasing the life of the drive)
      1. Three approaches are outlined here–in this guide I’ll reflect the GitHub page in saving reports to a temporary file system using tmpfs
        1. Add an entry to fstab
          1. echo "weewx_reports /var/weewx/reports tmpfs size=20M,noexec,nosuid,nodev 0 0" | sudo tee -a /etc/fstab
        2. Mount the new file system
          1. sudo mkdir -p /var/weewx/reports
          2. sudo mount -a
        3. Update weewx.config file to point to new directory
          1. sudo sed -i -e 's%HTML_ROOT =.*%HTML_ROOT = /var/weewx/reports%' /etc/weewx/weewx.conf
        4. Restart WeeWX service
          1. sudo service weewx restart
        5. Create symbolic link to point webserver to the reports
          1. sudo ln -s /var/weewx/reports /var/www/html/weewx
        6. Give the web server the ability to read from the directory
          1. sudo chmod -R 755 /var/www/html/weewx

At this point, go ahead and browse out to http://youripaddress/weewx/ to see your weather.

Notes:

WeeWX updates the webpage every 30 minutes (1800 seconds) out of the box.  You can force a report update by executing wee_reports weewx.conf or you can modify the /etc/weewx/weewx.conf file by changing the archive_interval variable (in seconds) under the [StdArchive] section.

You can modify the Weewx configuration by editing: /etc/weewx/weewx.conf

You can validate if WeeWX is running by executing: service weewx status

You can look at diagnostics logs by following the guide here: http://www.weewx.com/docs/usersguide.htm#monitoring

Best practices guide on using WeeWX + Raspberry PI: https://github.com/weewx/weewx/wiki/Raspberry%20Pi

How to upgrade your Windows Server Evaluation/Trial

Scenario: You downloaded the evaluation copy of Windows Server and you have 180 days to test out whatever you are working on.  Fast forward a few months and you only have a few days left and you are so happy with how it works, you go out and buy the whole license key.  When you go to apply the license key under System, you get a big ol’ error that says: “This edition cannot be upgraded.”

Solution:

You can use the DISM tool to figure out what versions of Windows Server you can upgrade to, and also use the tool to help change the product key of the version installed.

Easy enough, let’s go ahead and open up command prompt as an administrator (right click on windows flag/start icon, Command Prompt (Admin):

Execute the following command to find out what versions you can upgrade to:

Dism /Online /Get-TargetEditions

In this case, you can see I can upgrade to ServerStandard or ServerDatacenter

Next, let’s go ahead and actually upgrade the edition and inject my license key:

Dism /Online /Set-Edition:TheEditionListedYouWantToGoTo /AcceptEula /ProductKey:XXXXX-XXXXX-XXXXX-XXXXX-XXXXX

Viola!  At this point, I simply need to reboot and my instance will be upgraded accordingly.

[Tutorial] Integrate Visual Studio Code with Visual Studio Team Services

Here’s a quick way to integrate Visual Studio Code with Visual Studio Team Services.

  1. Create a new Team Project
    1. Instructions on how to create a new Team Project are outlined here: https://docs.microsoft.com/en-us/vsts/accounts/create-account-msa-or-work-student
  2. Create a Personal Access Token
    1. Instructions on how to generate a personal access token are outlined here: https://docs.microsoft.com/en-us/vsts/accounts/use-personal-access-tokens-to-authenticate
  3. Download and install Git: https://git-scm.com/download/
  4. Download Visual Studio Code: https://code.visualstudio.com/Download
  5. Inside of Visual Studio Code, click on the Extensions button
  6. Search the marketplace for Visual Studio Team Services and select Install button
  7. Once the extension has been Installed, click on the Reload button.
  8. Inside of Visual Studio Code, press F1 on your keyboard and type Git: Clone

  9. Once prompted, type in the URL to your Team Project and click on the Open Repository button
  10. Once in the repository, type: Team: Sign In  Select Provide an Access token manually, enter the Personal Access Token from Visual Studio Online and press Enter on your keyboard
  11. From there, go ahead and make a change to any of the files in your Repository
  12. Click on the Source Control icon in Visual Studio Code
  13. Select Commit All
    1. Note: You will be prompted to type in a commit message, go ahead and type in what you changed

  14. Either select Push from the button in the top right, or click the Push button in the bottom left corner
  15. Validate you see the committed changes in Visual Studio Team Services

Windows 10 – Missing Windows Disc Image Burner for ISO files

In Windows, you typically are able to download a .ISO file, right click on it, and burn it via your CD/DVD drive using the Windows Disc Image Burner application. Unfortunately, for whatever reason my machine is missing this menu item.

A quick workaround that doesn’t involve any registry hacks is to simply right click on the file, select Open With, and select Choose another app.

Select More apps and scroll to the bottom and select Look for another app on this PC.

Navigate to C:\Windows\System32, select isoburn.exe, and click Open

At this point, you can go ahead and burn your iso 🙂

 

[Tutorial] Gathering trace/event logs in ADFS v2.0 and v3.0

Problem:

Gathering trace/event logs in ADFS is not a trivial task.  The following article will show you how to gather these logs to further help investigate relying party trust issues or issues with end users authenticating to the service.  This tutorial will be leveraging ADFS v3.0 on Server 2012 R2.  The same steps should apply for v2.0 on Server 2008 R2.  This process does change slighting in ADFS on Server 2016 as the logging engine was rewritten.  Depending on demand, a second article will be released for ADFS on Server 2016.

Caviets:

Before beginning, as a side note, debugging in ADFS v2-3 is honestly a total PITA (pain in the… butt).  The problem with ADFS logging is logs are stored on the machines serving the requests, not centrally.  In this case, you will likely have to enable tracing on each ADFS server, or configure your load balancer/host file to temporarly route requests to a specific machine so you know which server to hunt down for the logs.  Likewise, as you will find at the end of the tutorial, the logs gathered from ADFS are very verbose.  Take some time to familiarize yourself with the logs of a working request vs a failure to get used to what logs are actually meaningful.

Tutorial:

Enable list of events/audits to be logged

  1. Login to one of your ADFS servers that you believe will be authenticating the end users
  2. Open Server Manager

  3. In Server Manager, select Tools -> AD FS Management
  4. In AD FS Management, select AD FS in the top left and select Edit Federation Service Properties…
  5. Click on the Events tab and check all the items you wish to log and click OK

Enable tracing

  1. Open Server Manager
  2. Select Tools -> Event Viewer
  3. In Event Viewer, select View in the top menu, and select Show Analytic and Debug Logs
  4. Expand Applications and Services Logs, expand AD FS Tracing, and select Debug
    1. Note: In ADFS v2, the AD FS Tracing folder will be called AD FS 2.0 Tracing
  5. When you are ready to begin collecting logs, right click on Debug and select Enable Log
  6. Click OK when prompted to write over the existing event logs
    1. Note: Each time you enable/disable AD FS Tracing, Event Viewer will purge your last results.  I highly recommend you export your logs if you need them for comparison at a later time.
  7. At this point, recreate the issue, error, or login to the relying party you want to debug.
  8. Once you have recreated the error or logged in, go back to Event Viewer, right click on Debug and select Disable Log
  9. At this point, you should have some events captured to further analyse 🙂
  10. Optional Step: Right click on Debug and select Save All Events As…  This will export to a evtx file, in which this can be sent to another team for analysis or you can reference the logs at a later time.
    1. Note: If you are sending the events over to another team for analysis, zip the logs as it will greatly decrease the file size 🙂

Common error when enabling Debug logging

One error I typically see is the following:

AD FS Debug – The requested operation cannot be performed over an enabled direct channel.  The channel must first be disabled before performing the requested operation

This error is caused by a misconfiguration on the logging properties of the Debug log.  Please verify that you have not manually enabled the debug log nor have the maximum log file size set to Overwrite events as needed.

To fix, right click on Debug and select Properties

Typically, the screenshot below is an example of the incorrect settings used; make sure that Enable Logging is unchecked and is Do not overwrite events ( Clear logs manually ) is checked

Here is a picture of the correct settings for the AD FS Tracing Debug Logs; at which point, once the settings are applied, you should no longer receive this error when conducting your debug/trace logging.

[Tutorial] Using Fiddler to debug SAML tokens on Mobile Devices (Android)

Use Case:

This guide will go over configuring Fiddler to intercept traffic from mobile devices for debugging purposes.  This scenario can be beneficial in tracing/debugging SAML tokens issued from your IdP for a mobile application to consume.  We will be able to validate all traffic flowing in/from the Android device.

Configuring/Setting up Fiddler:

  1. Grab the latest copy of Fiddler from their website for Windows (it is a free download)
    1. https://www.telerik.com/download/fiddlerDownload Fiddler
  2. Install Fiddler on your local machine
    1. Double click fiddlersetup.exe
      Run fiddlersetup
    2. Agree to the End User License Agreement
      Fiddler Install - Accept EULA
    3. Set the installation directory and click Install
      Fiddler Install - Destination Folder
    4. Close the setup wizard
      Fiddler Install - Close Installation
  3. Launch Fiddler
    Launch Fiddler - Windows 10
  4. Click Cancel if prompted about AppContainers
    Fiddler - AppContainer Configuration - Cancel
  5. With Fiddler open click on Tools -> Telerik Fiddler Options…
    Fiddler - Tools - Telerik Fiddler Options
  6. Click on the Connections tab and check Allow remote computers to connect

  7. You will receive a dialog box saying it will need to restart.  Click OK and close out of Fiddler
  8. Once you relaunch Fiddler, click on the down arrow (if shown) and hover over the Online icon

At this point, Fiddler is configured properly, let’s shift over to your mobile device. We’ll shift gears to configuring the Android device to push traffic to Fiddler.

Configuring an Android device
(Android v6.0.1 at the time of writing)

  1. Slide down the notifications drawer from the top of the screen and hit the Settings (gear) icon in the top right
  2. Select Wi-Fi under the Wireless and networks section
  3. Select the wireless network you are connected to and click Edit

  4. Scroll down and check Show advanced options

  5. Select the drop-down for Proxy and choose Manual

  6. Type in the IP address gathered from Fiddler for the Proxy host name and set the Proxy Port to 8888 and click Save
    1. Note: 8888 is the default port for Fiddler, the port can be found under Fiddler -> Telerik Fiddler Options -> Connections tab
  7. Next, open up your web browser and navigate to http://ipv4.fiddler:8888
    1. Note: This is a small webpage served by the Fiddler application to validate the proxy settings are correct.  Likewise, we will use this page in the next step for SSL decryption
  8. On the Fiddler Echo Service page, click on the You can download the FiddlerRoot Certificate link
    1. Note: This download Fiddler’s root certificate to allow us to intercept SSL traffic for debugging purposes
  9. Once the certificate has downloaded, type Fiddler as the Certificate name and click OK

  10. Optional step: Open up your web browser and navigate to a website using SSL (I did https://google.com)
    1. Note: Here you can validate that the SSL certificate used is Fiddler’s root certificate.  This is a good sign that we are intercepting the traffic

Turn off Fiddler from intercepting SSL traffic

Remove the proxy settings

  1. Slide down the notifications drawer from the top of the screen and hit the Settings (gear) icon in the top right
  2. Select Wi-Fi under the Wireless and networks section
  3. Select the wireless network you are connected to and click Edit

  4. Scroll down and check Show advanced options (you should see your old proxy settings unlike my screenshot below)

  5. Select the drop-down for Proxy and choose None

  6. Select Save
  7. At this point, you should be able to capture the traffic through the Fiddler application on your Windows machine; see the screenshot below showing traffic from the android device
    1. NOTE/TIP: If you turn off capturing, you will turn off capturing on Windows, but not for the mobile device.  This can help cut down on the “noise” in getting your sample/debug logs.

Remove the Fiddler SSL certificate

  1. Slide down the notifications drawer from the top of the screen and hit the Settings (gear) icon in the top right
  2. Select Security

  3. Select Trusted credentials

  4. Select the User tab on the Trusted credentials window
  5. Scroll down through the certificate information and towards the bottom you will see a Remove button; press the REMOVE button.
    1. Note: You have to scroll the text, there is no scrollbar until you start the scrolling gesture