HAMLIB SDK Version 3 Evolving the JT-ware Software Developer's Kit (JTSDK)

JTSDK 3.4.1 Questions and Suggestions for Resolving for Common Issues

What has happened to the original JTSDK creator?

Greg Beam KI7MT is still around and with us on the JTSDK Forum at jtsdk@groups.io. People's interests change; you will see from Greg's posts that he has evolved past active JTSDK development.

We cannot thank Greg more highly for his past and ongoing work - especially as a mentor at this point in time.

No politics here. This is not a push out or push aside. The 'hamlibdk' group as entities seek no glory nor recognition. Contributors just evolved work where Greg left off as the need was there for this to occur.

Other developers have gone silent (but may still be around in the background) due to some of the nastiness in our environment. 

Where can I ask question ?

The original JTSDK Forum jtsdk@groups.io is used as the primary communication point for this project.

Have no hesitations posting questions to there. If you are unable to post then email main contributors in that forum directly.

Windows 11: Deployments grind to a halt

UnderWindows 11 sometimes users have observed that the tasks appear to grind to a halt.

We have no answers to why this happens.

Press ENTER in the "stalled" Window and all seems to come back to life..

What happens if I receive a PowerShell security warning? Is it safe to run the PowerShell Scripts?

You may receive the following security notification:

Execution Policy Change
The execution policy helps protect you from scripts that you do not trust. Changing 
the execution policy might expose you to the security risks described in the 
about_Execution_Policies help topic at https:/go.microsoft.com/fwlink/?LinkID=135170. 
Do you want to change the execution policy?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"):

It is safe to select Y. These scripts are not any processor flaw tests. Remember that developing these scripts is also a learning exercise for the Development Team.

How does the SDK use PowerShell? Why does the SDK need to use PowerShell when previous kits used CMD scripts?

The second part first: PowerShell creates a HIGHLY EXTENSIBLE scripted environment. The PowerShell environment is much more powerful and offers facilities that vastly eclipse capabilities of the old CMD.EXE processor and its scripts. Many tasks became onerous and were pushing the boundaries of what CMD.EXE-capable scripts could perform. Some tasks (such as extracting version information from the cmake files) became near impossible with CMD.EXE scripts. Therefore the PowerShell 'universal' scripting environment - provided by Microsoft STANDARD with Windows - was selected as the investigation tool for this project.

The basic concept of supporting Windows Environment Variables through PowerShell $env: facilities ease access into CMake, QMake and the MinGW compilers - components that will always remain a cornerstone concepts.

How do I upgrade existing JTSDK deployments?

The JTSDK64-3.4.1 should be able to be installed over the top of previous kits - yet it is highly recommended that you start from scratch.

It is highly recommended that kits are deployed into Virtual machines (see Installation section).

It is highly recommended that a new, fresh deployment be considered with each new base package release.

You can keep old deployment(s) by closing all open environment windows and renaming the deployment folder (i.e. C:\JTSDK64-Tools to C:\JTSDK64-Tools-Old)

If you need to revert back to your old deployment then all you need do is rename that folder back to its original filename (i.e. C:\JTSDK64-Tools-Old back to C:\JTSDK64-Tools)

How do I update the JTSDK?

Maintenance updates will be applied in the form of "Tools" packages. These packages are designed to be deployed to existing "Base" packages.

Where do I get and how do I apply 'Update'/'Patch'  packages?

Update/Patch  packages can be downloaded from https://sourceforge.net/projects/hamlib-sdk/files/Windows/JTSDK-4.0-Stream/ .

These steps assume that you have a deployed base environment

• Download any Update/Patch packages if they exist ( i.e. packages with a "U" in the filename) from https://sourceforge.net/projects/hamlib-sdk/files/Windows/JTSDK-4.0-Stream/.
• Deploy the tools package to your JTSDK64 install directory.

Updates may apply to the MSYS2 environment. Therefore the "profile" directory for MSYS2 should be deleted and re-created.

• (Optional) Copy any folders stored under x:\JTSDK64-Tools\tools\msys64\home to a backup location. This should enable you to access previous work.
• Close any open MSYS2 environment terminal(s).
• Perform the system upgrade.
• Delete folders that exist under x:\JTSDK64-Tools\tools\msys64\home.
• i.e: PowerShell: Remove-Item "C:\JTSDK64-Tools\tools\msys64\home*" -Recurse -Force
• Re-open MSYS2 with the msys2 command for maintenance or mingw64 for development work inside the PowerShell terminal

Note: The src directory in your backup should contain your previous work. It is safe to copy this enture folder across to your new MSYS2 Profile.

You will need to pull down your Hamlib repository again and/or restore any custom work to folders created within the MSYS2 environment. See Step 3b below.

Issues and Potential Work Arounds

Environments have become EXTREMELY COMPLEX. What can work on one machine may not work on another.

These notes may assist you getting around some of the issues that have been observed.

VC 2022 Runtimes Deployment Error during postinstall

When you start up the JTSDK64-Setup environment in Windows 11 you may notice the following:

-------------------------------------------
           JTSDK Setup v3.4.1
-------------------------------------------

  Required Tools

     PowerShell ..... 5.1.22621.2506
     VC Runtimes .... Not Installed
     Git ............ Installed
     OmniRig ........ Installed

  Qt Script-Provisioned Tool Chains

    x64:
     MinGW 8.1 ...... Qt Not Installed
     MinGW 11.x ..... Qt Not Installed

    x86 (Legacy: unsupported):
     MinGW 8.1 ...... Qt Not Installed

  Optional Components

     VS Code ........ Installed
     Boost .......... Not Installed

  Post Install / Manual Setup Commands

     Main Install ... postinstall
     MSYS2 Shell .... msys2
	 
PS C:\JTSDK64-Tools>

In the above example it is shown that the VC 2022 Runtimes are not deployed.

When you run postinstall with deployment of VC 2022 Runtimes selected for deployment the following error occurs:

...
 Enter Your Install/Redeployment Selection(s):

(required) VC/C++ Runtimes (Y|N) .: y
...
...
-----------------------------------------------------
Installing Visual C/C++ x86 and x64 Runtimes
-----------------------------------------------------

* Downloading VC/C++ Installer

  --> Downloading the latest VC/C++ x86 Installer: Complete
  --> Downloading the latest VC/C++ x64 Installer: Complete

-----------------------------------------------------
 Visual C/C++ Runtime - Error In x86 Install
-----------------------------------------------------

****************************************************
Processing Error
****************************************************

The exit status from step [ VC Runtimes Install ] returned
a non-zero status. Check the error message and
and try again.

If the problem presists, contact: JTSDK@Groups.io

PS C:\JTSDK64-Tools\tools\setup\vcruntime>

This is because LATER versions of the VC Runtimes may be reployed in your Windows 11 deployment

The postinstall script has difficulty handling and detecting later versions of the VC runtimes.

This can be resolved with the following simple procedure:

• Restart the deployment.
• Ignore the deployment of The VC 2022 runtimes (i.e. select 'N') if you experience this error.

Wrong Qt deployment settings

By far the most common issue observed has occurred when end users have performed default deploys of  Qt 5.12.12 under past kits. Qt 5.12.x is no longer supported.

Qt 5.15.2 is the default version of Qt used for development.

Most development and testing has been performed with Qt 5.15.2 and its MinGW 8.1 environment.

All libraries and software including Boost, Hamlib and the JT-ware should be rebuilt if not using Qt 5.15.2.

The marker file to set the version of Qt used is found in x:\JTSDK64-Tools\config i.e. qt5.15.2 (as default).

Rename that file as appropriate to match the version of Qt that is deployed or that you want to use i.e. from qt5.15.2 to qt6.7.2  . 

Qt 5.15.2 Not Being Deployed

As of March 2024 it has been observed that Qt 5.15.2 is no longer available from "Mainstream" installer sources

Qt 5.15.2 is still available from "Archive" ast the time of writing. This means that Qt must be deployed manually from "Archive" Repositories

It is now recommended that one performs a "Full" deployment of Qt at install time. 

This should properly deploy Qt 6.7.2 along with the Qt Maintenance Tool. 

Once the JTSDK is deployed you can then add Qt 5.15.2 MANUALLY from Archive. 

See the guide https://sourceforge.net/projects/hamlib-sdk/files/JTware-Deployment/Deploying-Archived-Versions-of-Qt-via-Maintenance-Tool.docx for latest instructions

Issues observed when trying to build software with Qt 6.7.2 deployments

Qt 6 is not currently supported by the JT-ware developers. Current source code of all known JT-ware forks at the time that this document was written will not compile under Qt 6.

JT-ware developers are researching Qt 6-compatible code conversion. 

Issues observed when trying to perform a Full Deployment: Qt 5 and/or Qt 6 does not Deploy

JTSDK 2.3.2 with Tools 3.2.3.3 Has the capability to deploy two "streams" of Qt kit:

• Default Deployment: deploys Qt Version 5.15.2. This is not likely to change - but at some stage it may be withdrawn from release by the Qt maintainers. When this occurs our kit will not be able to find these files; it will not install.
• Full Deployment: deploys both Qt versions 5.15.2 and Qt 6.7.2 at the time that this document was written. If one or either of these versions cannot be found then these kits will not be installed.

This kit by default deploys the Qt SDK For Open Source Release. This is also compliant with the basic beliefs, method and release licensing terms for WSJTX.

These Qt deployment repositories are stored in geo-assigned download repositories. Each repository ONLY MAKES THE LATEST DOWNLOAD RELEASE STREAMS AVAILABLE to the Qt installer.

The Qt maintainers may at some stage in the future bump the versions available i.e. From 6.7.2 to  6.7.2 (as an example). 

Our kit has a way of dealing with this:

Refer to the file x:\JTSDK64-Tools\config\Versions.ini 

There are two keys that (at the time of writing) can br found at the bottom of the file:

File: x:\JTSDK64-Tools\config\Versions.ini
...
#future enhancement to set Qt version
qt5v=5.15.2
qt6v=6.7.2

• Edit the appropriate key (most likely qt6v) to point to the currently distributable Qt version.
• Save your new Versions.ini file.
• Close any open environment terminals (Most likely to be a JTSDK64-Setup terminal).
• Restart the appropriate environment terminal. Use the postinstall script to attempt a fresh scripted deployment of Qt.

JT-ware compile error around code that uses the Resource Compilers

Many users have observed a compiler error (especially with JTDX) at around the 60% mark that relates to the Resource Compiler.

[ 14%] gcc.exe:Building Fortran object CMakeFiles/wsjt_fort_omp.dir/lib/chkmsg.f90.obj
error: C:JTSDK64-Toolstmpwsjtx-outputqt5.15.22.5.4ReleasebuildVersionResource_wsprd.rc: 
No such file or directory
gcc.exe: warning: '[ 14%] Building Fortran object CMakeFiles/wsjt_fort.dir/lib/chkmsg.f90.obj
-x c' after last input file has no effect
gcc.exe: fatal error: no input files
compilation terminated.
/usr/bin/windres: preprocessing failed.

This is a conflict caused by the complexity of resources i.e. Mixing of Qt-supplied MinGW and MSYS2-supplied tools/compilers.

Resolution: Rename MSYS2 windres.exe

• Delete the source and build folders in C:\JTSDK64-Tools\tmp (use the Windows File Manager as its easier than PowerShell Commands)
  
  
  • For the brave, enter the following commands at a JTSDK64-Tools Command Prompt:
  
  
Set-Location -Path "C:\JTSDK64-Tools\tmp"

Get-ChildItem -Recurse -Filter "wsjtx*" | Remove-Item -Force -Recurse     <== BE CAREFUL: This can be as dangerous as a Unix rm -rf * if executed in the wrong path.


• Execute the following command at the PowerShell prompt:


Rename-Item -Path "C:\JTSDK64-Tools\tools\msys64\usr\bin\windres.exe" -NewName "windres.exe.disabled"


• - Re-start the JT-ware build process.

Thanks for the reminder Tom (and Mike) :-)

Minor packaging issue observed when building JTDX

A CPACK issue may be observed when the JTSDK is deployed to a FRESH Windows OS.

...
CPack: - Run preinstall target for: jtdx
CPack: - Install project: jtdx []
warning: target 'libhamlib-4.dll' is not absolute...
warning: target 'libhamlib-4.dll' does not exist...
CMake Error at C:/JTSDK64-Tools/tmp/wsjtx/CMake/Modules/GetPrerequisites.cmake:817 (message):
  C:/JTSDK64-Tools/tools/Qt/Tools/mingw810_64/bin/objdump.exe failed: 1

  C:/JTSDK64-Tools/tools/Qt/Tools/mingw810_64/bin/objdump.exe:
  'libhamlib-4.dll': No such file

Call Stack (most recent call first):
  C:/JTSDK64-Tools/tmp/wsjtx/CMake/Modules/GetPrerequisites.cmake:942 (get_prerequisites)
  C:/JTSDK64-Tools/tmp/wsjtx/CMake/Modules/BundleUtilities.cmake:585 (get_prerequisites)
  C:/JTSDK64-Tools/tmp/wsjtx/CMake/Modules/BundleUtilities.cmake:804 (get_bundle_keys)
  C:/JTSDK64-Tools/tmp/wsjtx-output/qt/5.15.2/2.2.0/Release/build/cmake_install.cmake:153 (fixup_bundle)

CPack Error: Error when generating package: jtdx
mingw32-make.exe: *** [Makefile:112: package] Error 1

--------------------------------------------
 WINDOWS INSTALLER BUILD ERROR
--------------------------------------------

 There was a problem building the package,
 or the script could not find:

 C:\JTSDK64-Tools\tmp\wsjtx-output\qt\5.15.2\2.2.0\Release\build\

 Check the Cmake logs for any errors, or
 correct any build script issues that were
 obverved and try to rebuild the package.

PS C:\JTSDK64-Tools\tmp\wsjtx-output\qt\5.15.2\2.2.0\Release\build>

Issue: Unable to package JTDX because of NUMEROUS DLL's reported as being missing

Resolution: Issue an appropriate command to find the path of necessary libraries before performing a jtbuild package command

• If you are using Qt 5.15.2: 

$env:Path += ";C:\Windows\SysWOW64\downlevel;C:\JTSDK64-Tools\tools\hamlib\qt\5.15.2\bin"

• If you are using Qt 6.7.2: 

$env:Path += ";C:\Windows\SysWOW64\downlevel;C:\JTSDK64-Tools\tools\hamlib\qt\6.7.2\bin"

These paths can be added to the main Windows system paths.

• Control Panel/About/Advanced System Settings/Advanced/Environment Variables.
• Append the lines to the existing local user's path:
• For Qt 5.15.2:

C:\Windows\SysWOW64\downlevel;C:\JTSDK64-Tools\tools\hamlib\qt\5.15.2\bin


• For Qt 6.7.2:

C:\Windows\SysWOW64\downlevel;C:\JTSDK64-Tools\tools\hamlib\qt\6.7.2\bin


• Restart the build proess i.e. jtbuild package

The issue will NOT be coded into a fix at this stage for the JTSDK as it may upset something else.

WSJT-X does NOT demonstrate the issue. JTDX does. Therefore in all probability the issue is with JTDX and its CMAKE scripts. Further reports to the JTDX forum if it is observed may assist in resolution. A direct report of this issue has already been made to JTDX Developers.

Ref: https://groups.io/g/JTSDK/message/1779

Anomalies that do not make sense

Note:This issue is different to the one described before this. Review messages carefully.

During development and testing quite a number of issues were observed that were VERY FRUSTRATING and DID NOT MAKE ANY SENSE.

An example is provided below within JTDX where a deployment has been performed, a JTDX build successfully completed (and packaged), yet after a system reboot the following error was observed (again in CPACK):

...
CPack: - Install project: jtdx []
CMake Error at C:/JTSDK64-Tools/tmp/wsjtx/CMake/Modules/GetPrerequisites.cmake:817 (message):
  C:/JTSDK64-Tools/tools/Qt/Tools/mingw810_64/bin/objdump.exe failed: 1

Call Stack (most recent call first):
  C:/JTSDK64-Tools/tmp/wsjtx/CMake/Modules/GetPrerequisites.cmake:942 (get_prerequisites)
  C:/JTSDK64-Tools/tmp/wsjtx/CMake/Modules/GetPrerequisites.cmake:942 (get_prerequisites)
  C:/JTSDK64-Tools/tmp/wsjtx/CMake/Modules/GetPrerequisites.cmake:942 (get_prerequisites)
  C:/JTSDK64-Tools/tmp/wsjtx/CMake/Modules/BundleUtilities.cmake:585 (get_prerequisites)
  C:/JTSDK64-Tools/tmp/wsjtx/CMake/Modules/BundleUtilities.cmake:804 (get_bundle_keys)
  C:/JTSDK64-Tools/tmp/wsjtx-output/qt/5.15.2/2.2.0/Release/build/cmake_install.cmake:153 (fixup_bundle)


CPack Error: Error when generating package: jtdx
mingw32-make.exe: *** [Makefile:112: package] Error 1

--------------------------------------------
 WINDOWS INSTALLER BUILD ERROR
--------------------------------------------
...

Possible Cause: it is more than likely that a deployed MSYS2 utility is somehow conflicting with the set of utilities provided by Qt's MinGW Compiler set. (Ed. Note: This proved to be the case in this instance).

Test for this Issue: Disable path access to the MSYS2 Tools from the JTSDK64-Tools Environment.

• Edit C:\JTSDK64-Tools\config\Versions.ini
• Change key unixtools=enabled to unixtools=disabled
• Save and exit.
• Close all MSYS2 and Powershell Terminals
• Re-launch the JTSDK64-Tools environment.
• Re-attempt the build.

If the build succeeds then it is more than likely that SOMEHOW a MSYS2/mingw64 etc. tool is conflicting.

At this point you should review changes made to your environment and reset the unixtools key to unixtools=enabled .

Should all self-diagnosis attempts fail then seek further advice at the JTSDK Forum.

Packaging Issues: Support for an "extras" folder

In order to overcome several issues listed above an extras folder has been included into X:\JTSDK64-Tools\tools where additional "missing" packaged DLL's can be placed.

i.e. copy missing DLL's, such as libgfortran-4.dll into this folder for recent WSJTX-builds to succeed.

Packaging requirements are therefore met.

This option is NOT RECOMMENDED FOR LONG TERM USE. It should be used as a LAST RESORT.

Fix the actual problem - whether it be how the libraries used are built OR it may be redundant scripted requirements from legacy code that has been re-introduced.

Conclusion and Further References

The aim of JTSDK64-Tools is to use an Agile delivery approach to create a high-quality, yet flexible build system.

Base ref: https://sourceforge.net/projects/hamlib-sdk/files/Windows/JTSDK-4.0-Stream/README.md