Building P4P against PVXS+TLS #177

ernestow · 2025-06-11T22:56:19Z

Changes made in order to compile P4P with the forked PVXS tls branch.

HOST OS: Rocky 9
Python ENV 3.10

Due to conflicts with openssl library provided by conda at run time I needed the following:
LD_PRELOAD=/usr/lib64/libssl.so:/usr/lib64/libcrypto.so LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH bin/rhel9-x86
_64/pvagw -h

Here is my CONFIG_SITE.local file for reference:

(p4p-build-env) ernesto@dev-epicsgw $ cat CONFIG_SITE.local
# Enable OpenSSL support
USR_CPPFLAGS += -DPVXS_ENABLE_OPENSSL

# CONDA ENV has openssl but version is older than the system library from Rocky 9, which is what PVXS used.
# CONDA_PREFIX=/sdf/sw/epics/package/anaconda/2024.10/envs/p4p-build-env

# ========================================================
# We will use openssl from the Rocky 9 Linux Distribution:
# This will make P4P consistent with PVXS when linking.
# ========================================================
OPENSSL_LIB=/usr/lib64
OPENSSL_INCLUDE=/usr/include/openssl

# =================================================
# Build P4P with TLS support.
# =================================================
PVXS_WITH_TLS = YES

# =============================
# Build Application Statically
# =============================
#SHARED_LIBRARIES=NO
#STATIC_BUILD=YES
(p4p-build-env) ernesto@dev-epicsgw $

md: edit to be less shouty

mdavidsaver · 2025-06-11T23:21:09Z

@ernestow The change made by this PR should not be necessary with epics-base/pvxs#53 or epics-base/pvxs#115. Some added Makefile magic in PVXS should automatically inject a dependency on libopenssl in the same way that the dependency on libevent_core is already added.

This mechanics keys off of the LIBEVENT_$(T_A) and OPENSSL_$(T_A) Make variables. If you openssl install has a non-standard prefex, then either:

make OPENSSL_linux-x86_64=/some/other/usr

Or append to configure/CONFIG_SITE.local.

After building, look for the results in cfg/TOOLCHAIN_PVXS.*:

$ grep . cfg/TOOLCHAIN_PVXS.*
cfg/TOOLCHAIN_PVXS.linux-x86_64:OPENSSL_PREFIX_linux-x86_64 = 
cfg/TOOLCHAIN_PVXS.linux-x86_64:LIBEVENT_PREFIX_linux-x86_64 = 
cfg/TOOLCHAIN_PVXS.linux-x86_64:LIBEVENT_BUNDLE_LIBS_linux-x86_64 = event_openssl event_core event_pthreads
cfg/TOOLCHAIN_PVXS.linux-x86_64:LIBEVENT_SYS_LIBS_linux-x86_64 = ssl crypto

mdavidsaver · 2025-06-11T23:25:27Z

make OPENSSL_linux-x86_64=/some/other/usr

This variable is set when building PVXS (not P4P).

ernestow · 2025-06-12T03:21:41Z

After,
make OPENSSL_rhel9-x86_64=/usr/lib64
OPENSSL_PREFIX_rhel9-x86_64 is still empty.

Could be because you auto discover openssl via: "configure/probe-openssl.c" ?

ernesto@dev-epicsgw (tls) $ cat cfg/TOOLCHAIN_PVXS.rhel9-x86_64
OPENSSL_PREFIX_rhel9-x86_64 =
LIBEVENT_PREFIX_rhel9-x86_64 =
LIBEVENT_BUNDLE_LIBS_rhel9-x86_64 = event_openssl event_core event_pthreads
LIBEVENT_SYS_LIBS_rhel9-x86_64 = ssl crypto

Add OpenSSL library paths for specific targets

USR_LDFLAGS_windows-x64-mingw += -L/usr/x86_64-w64-mingw32/lib64

ernestow · 2025-06-12T03:41:14Z

At runtime, I get:
bin/rhel9-x86_64/pvagw -h
Traceback (most recent call last):
File "/sdf/group/cds/sw/epics/users/TestStand/modules/p4p/p4p-tls/bin/rhel9-x86_64/pvagw", line 9, in
from p4p.gw import main
File "/sdf/group/cds/sw/epics/users/TestStand/modules/p4p/p4p-tls/bin/rhel9-x86_64/../../python3.10/rhel9-x86_64/p4p/init.py", line 14, in
from .wrapper import Value, Type
File "/sdf/group/cds/sw/epics/users/TestStand/modules/p4p/p4p-tls/bin/rhel9-x86_64/../../python3.10/rhel9-x86_64/p4p/wrapper.py", line 5, in
from . import _p4p
ImportError: /sdf/group/cds/sw/epics/users/TestStand/modules/p4p/p4p-tls/bin/rhel9-x86_64/../../python3.10/rhel9-x86_64/p4p/_p4p.so: undefined symbol: OCSP_RESPONSE_new

So, then I tried:
LD_PRELOAD=/usr/lib64/libssl.so:/usr/lib64/libcrypto.so LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH bin/rhel9-x86_64/pvagw -h
Traceback (most recent call last):
File "/sdf/group/cds/sw/epics/users/TestStand/modules/p4p/p4p-tls/bin/rhel9-x86_64/pvagw", line 9, in
from p4p.gw import main
File "/sdf/group/cds/sw/epics/users/TestStand/modules/p4p/p4p-tls/bin/rhel9-x86_64/../../python3.10/rhel9-x86_64/p4p/init.py", line 14, in
from .wrapper import Value, Type
File "/sdf/group/cds/sw/epics/users/TestStand/modules/p4p/p4p-tls/bin/rhel9-x86_64/../../python3.10/rhel9-x86_64/p4p/wrapper.py", line 5, in
from . import _p4p
ImportError: /sdf/group/cds/sw/epics/users/TestStand/modules/p4p/p4p-tls/bin/rhel9-x86_64/../../python3.10/rhel9-x86_64/p4p/_p4p.so: undefined symbol: evbuffer_new

ernestow · 2025-06-14T06:58:15Z

Following Michael's advice on how to include required libraries from PVXS.
I will revoke this one and do a new pull request

ernestow · 2025-06-14T07:01:11Z

Sorry, I am re-opening the pull request and will simply update.

ernestow · 2025-06-14T07:09:14Z

“Updated the patch based on review — now using dependencies from "<>/pvxs/cfg/" as suggested. Let me know if further changes are needed.”

By the way at run time, I still have the issue with interference python 3.10 and openssl.
Thus, my need to execute with LD_PRELOAD:
LD_PRELOAD=/usr/lib64/libssl.so:/usr/lib64/libcrypto.so LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH bin/rhel9-x86_64/pvagw -h

mdavidsaver · 2025-06-16T04:21:32Z

@ernestow I am changing this PR to a Draft. Ultimately, the SPVA changes will not require any change to dependent module builds. At present this is not the case with epics-base/pvxs#115 .

mdavidsaver · 2025-06-16T04:40:15Z

Also, I take it that you are not setting LINKER_USE_RPATH?

It might be interesting to see how your _p4p.so is built. (eg. a debian:12 container build)

$ readelf -d python3.11/linux-x86_64/p4p/_p4p.so 

Dynamic section at offset 0x5cb30 contains 30 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libpvxs.so.1.3]
 0x0000000000000001 (NEEDED)             Shared library: [libCom.so.3.24.1]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [_p4p.so]
 0x000000000000001d (RUNPATH)            Library runpath: [/opt/epics-base/lib/linux-x86_64:/opt/pvxs/lib/linux-x86_64:/opt/p4p/lib/linux-x86_64:/usr/lib/x86_64-linux-gnu]
...

$ ldd python3.11/linux-x86_64/p4p/_p4p.so 
        linux-vdso.so.1 (0x00007f1663ce4000)
        libpvxs.so.1.3 => /opt/pvxs/lib/linux-x86_64/libpvxs.so.1.3 (0x00007f1663aba000)
        libCom.so.3.24.1 => /opt/epics-base/lib/linux-x86_64/libCom.so.3.24.1 (0x00007f1663a3a000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f1663800000)
        libgcc_s.so.1 => /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1663a1a000)
        libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6 (0x00007f166361f000)
        libevent_openssl-2.1.so.7 => /lib/x86_64-linux-gnu/libevent_openssl-2.1.so.7 (0x00007f1663610000)
        libevent_core-2.1.so.7 => /lib/x86_64-linux-gnu/libevent_core-2.1.so.7 (0x00007f16635da000)
        libevent_pthreads-2.1.so.7 => /lib/x86_64-linux-gnu/libevent_pthreads-2.1.so.7 (0x00007f16635d5000)
        libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x00007f166352c000)
        libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007f1663000000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f1663ce6000)
        libreadline.so.8 => /lib/x86_64-linux-gnu/libreadline.so.8 (0x00007f16634d2000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1662f20000)
        libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (0x00007f166349f000)

ernestow · 2025-06-16T16:07:17Z

I have removed the hack of using LD_PRELOAD at run time :)

Wordy comment below:

How to build P4P with the new PVXS which has TLS support (06/16/25)

Note:
There is a newer function (PKCS12_create_ex2) added to OpenSSL to:

-- Provide more fine-grained control for creating PKCS#12 certificate bundles (i.e., .p12 files)

-- Support explicit memory context and property query support (part of modern OpenSSL 3.x architecture)

Although creating certicates is the job of pvacms, that function is linked in the library via pvxs.
At run time a mistmatch between Host and conda openssl libraries causes the following issue:

===============================================================================================================
(p4p-build-env) ernesto@dev-epicsgw $ which openssl
/sdf/sw/epics/package/anaconda/2024.10/envs/p4p-build-env/bin/openssl
(p4p-build-env) ernesto@dev-epicsgw $ openssl version
OpenSSL 3.0.16 11 Feb 2025 (Library: OpenSSL 3.0.16 11 Feb 2025)
(p4p-build-env) ernesto@dev-epicsgw $ bin/rhel9-x86_64/pvagw -h
Traceback (most recent call last):
File "/sdf/group/cds/sw/epics/users/TestStand/modules/p4p/R4.2.0/bin/rhel9-x86_64/pvagw", line 9, in
from p4p.gw import main
File "/sdf/group/cds/sw/epics/users/TestStand/modules/p4p/R4.2.0/bin/rhel9-x86_64/../../python3.10/rhel9-x86_64/p4p/init.py", line 14, in
from .wrapper import Value, Type
File "/sdf/group/cds/sw/epics/users/TestStand/modules/p4p/R4.2.0/bin/rhel9-x86_64/../../python3.10/rhel9-x86_64/p4p/wrapper.py", line 5, in
from . import _p4p
ImportError: /sdf/group/cds/sw/epics/users/TestStand/modules/p4p/R4.2.0/bin/rhel9-x86_64/../../python3.10/rhel9-x86_64/p4p/_p4p.so: undefined symbol: PKCS12_create_ex2

Solution is choose a compatible version of openssl when building the conda environment for p4p:

First create a conda environment named ("p4p-build-env_tls-1.3") like so;

conda create -n p4p-build-env_tls-1.3 python=3.11
openssl=3.5.0 pip numpy Cython nose2 ply
-c conda-forge

conda env list |grep p4p
p4p-build-env /sdf/sw/epics/package/anaconda/2024.10/envs/p4p-build-env
p4p-build-env_tls-1.3 /sdf/sw/epics/package/anaconda/2024.10/envs/p4p-build-env_tls-1.3

Location:
/sdf/group/cds/sw/epics/users/TestStand/modules/p4p/p4p-tls
add a "CONFIG_SITE.local" to enable OPENSSL with TLS
/sdf/group/cds/sw/epics/users/TestStand/modules/p4p/p4p-tls/configure/CONFIG_SITE.local
-add a "RELEASE.local" to define location of EPICS BASE and PVXS
update src/Makefile [There is a pending pull request to the maintainer; so feature releases want need to do this]
In order to build P4P on dev-epicsgw:
step 1: source /sdf/group/cds/sw/epics/users/ernesto/sandbox/ML_Physics/conda-local-activate.sh p4p-build-env
step 2: cd /sdf/group/cds/sw/epics/users/TestStand/modules/p4p/p4p-tls
step 3: source /sdf/group/cds/sw/epics/users/ernesto/sandbox/ML_Physics/conda-local-deactivate.sh p4p-build-env_tls-1.3
step 4: make distclean; make
step 5: source /sdf/group/cds/sw/epics/users/ernesto/sandbox/ML_Physics/conda-local-deactivate.sh p4p-build-env

Try it with
/sdf/group/cds/sw/epics/users/TestStand/modules/p4p/p4p-tls/bin/rhel9-x86_64/pvagw -h

Results

usage: pvagw [-h] [--no-ban-local] [-v] [--logging LOGGING] [--debug] [-T] [--example-config] [--example-systemd]
config

positional arguments:
config Config file

options:
-h, --help show this help message and exit
--no-ban-local Legacy option. Ignored
-v, --verbose Enable basic logging with DEBUG level
--logging LOGGING Use logging config from file (JSON in dictConfig format)
--debug Enable extremely verbose low level PVA debugging
-T, --test-config Read and validate configuration files, then exit w/o starting a gateway. Also prints the names of
all configuration files read.
--example-config Write an example configuration file and exit. "--example-config -" writes to stdout
--example-systemd Write an example systemd unit file and exit "--example-systemd -" writes to stdout
(p4p-build-env_tls-1.3) ernesto@dev-epicsgw $

ernestow · 2025-06-16T16:09:00Z

My readelf looks good now, I think:

ernesto@dev-epicsgw $ readelf -d python3.11/rhel9-x86_64/p4p/_p4p.so

Dynamic section at offset 0x20d540 contains 39 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libCom.so.3.24.1]
0x0000000000000001 (NEEDED) Shared library: [libssl.so.3]
0x0000000000000001 (NEEDED) Shared library: [libcrypto.so.3]
0x0000000000000001 (NEEDED) Shared library: [libevent_openssl-2.1.so.7]
0x0000000000000001 (NEEDED) Shared library: [libevent_core-2.1.so.7]
0x0000000000000001 (NEEDED) Shared library: [libevent_pthreads-2.1.so.7]
0x0000000000000001 (NEEDED) Shared library: [libreadline.so.8]
0x0000000000000001 (NEEDED) Shared library: [libncurses.so.6]
0x0000000000000001 (NEEDED) Shared library: [libtinfo.so.6]
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x0000000000000001 (NEEDED) Shared library: [ld-linux-x86-64.so.2]
0x000000000000000e (SONAME) Library soname: [_p4p.so]
0x000000000000000f (RPATH) Library rpath: [/sdf/group/cds/sw/epics/users/TestStand/modules/p4p/p4p-tls/lib/rhel9-x86_64:/sdf/group/cds/sw/epics/users/TestStand/base/base-george/lib/rhel9-x86_64:/sdf/group/cds/sw/epics/users/TestStand/modules/pvxs/ernesto/lib/rhel9-x86_64:/sdf/sw/epics/package/anaconda/2024.10/envs/p4p-build-env_tls-1.3/lib]
0x000000000000000c (INIT) 0x3d000
0x000000000000000d (FINI) 0x1b159c
0x0000000000000019 (INIT_ARRAY) 0x2077d8
0x000000000000001b (INIT_ARRAYSZ) 256 (bytes)
0x000000000000001a (FINI_ARRAY) 0x2078d8
0x000000000000001c (FINI_ARRAYSZ) 8 (bytes)
0x000000006ffffef5 (GNU_HASH) 0x298
0x0000000000000005 (STRTAB) 0xc018
0x0000000000000006 (SYMTAB) 0x1e90
0x000000000000000a (STRSZ) 84102 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000003 (PLTGOT) 0x20e000
0x0000000000000002 (PLTRELSZ) 26280 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x35ab0
0x0000000000000007 (RELA) 0x21818
0x0000000000000008 (RELASZ) 82584 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000006ffffffe (VERNEED) 0x21618
0x000000006fffffff (VERNEEDNUM) 6
0x000000006ffffff0 (VERSYM) 0x2089e
0x000000006ffffff9 (RELACOUNT) 2357
0x0000000000000000 (NULL) 0x0
ernesto@dev-epicsgw $

mdavidsaver · 2025-06-23T16:40:27Z

@ernestow There is difference between our build situations. (not right or wrong, just different)

In my case, _p4p.so is linked against libpvxs.so (the shared library). In your case it is linked against libpvxs.a (the static library).

ernestow · 2025-06-23T16:55:14Z

That indeed was the issue: I built shared library for pvxs instead of only static: (p4p-build-env_tls-1.3) ***@***.*** $ pwd /sdf/group/cds/sw/epics/users/TestStand/modules/pvxs/latest/lib/rhel9-x86_64 (p4p-build-env_tls-1.3) ***@***.*** $ ls libpvxs.a libpvxsIoc.a libpvxsIoc.so@ libpvxsIoc.so.1.3* libpvxs.so@ libpvxs.so.1.3* Works nicely now: No changes, to P4P except I create a python environment with a newer version of openssl (p4p-build-env_tls-1.3) ***@***.*** $ bin/rhel9-x86_64/pvagw -h usage: pvagw [-h] [--no-ban-local] [-v] [--logging LOGGING] [--debug] [-T] [--example-config] [--example-systemd] config positional arguments: config Config file options: -h, --help show this help message and exit --no-ban-local Legacy option. Ignored -v, --verbose Enable basic logging with DEBUG level --logging LOGGING Use logging config from file (JSON in dictConfig format) --debug Enable extremely verbose low level PVA debugging -T, --test-config Read and validate configuration files, then exit w/o starting a gateway. Also prints the names of all configuration files read. --example-config Write an example configuration file and exit. "--example-config -" writes to stdout --example-systemd Write an example systemd unit file and exit "--example-systemd -" writes to stdout

…

________________________________ From: mdavidsaver ***@***.***> Sent: Monday, June 23, 2025 9:40 AM To: epics-base/p4p ***@***.***> Cc: Williams Jr., Ernest L. ***@***.***>; Mention ***@***.***> Subject: Re: [epics-base/p4p] Building P4P against PVXS+TLS (PR #177) BEWARE: This email originated outside of our organization. DO NOT CLICK links or attachments unless you recognize the sender and know the content is safe. [https://avatars.githubusercontent.com/u/4429215?s=20&v=4]mdavidsaver left a comment (epics-base/p4p#177)<#177 (comment)> @ernestow<https://github.com/ernestow> There is difference between our build situations. (not right or wrong, just different) In my case, _p4p.so is linked against libpvxs.so (the shared library). In your case it is linked against libpvxs.a (the static library). — Reply to this email directly, view it on GitHub<#177 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AB44BJESHHGDK3UZ5K4GOXD3FAUZFAVCNFSM6AAAAAB7D2IMTSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOJXGE3DGMRVHA>. You are receiving this because you were mentioned.

ernestow · 2025-06-25T07:52:40Z

Still not should how to use rpath correctly.
For now, I just ensure that I include the appropriate version of openssl in my conda environment when building P4P with make.
I still think that is a valid site-specific option. Do you agree?

mdavidsaver · 2025-06-25T16:15:00Z

I still think that is a valid site-specific option. Do you agree?

I don't disagree :) I don't work with conda, and so don't know what its developers consider the Right Way.

ernestow added 2 commits June 11, 2025 14:53

add support for PVXS with tls

e0cb3a9

add support for PVXS with tls

9de0705

mdavidsaver added the enhancement label Jun 11, 2025

mdavidsaver marked this pull request as draft June 11, 2025 23:10

ernestow added 3 commits June 11, 2025 21:22

fix typo in conditional logic for PVXS_WITH_TLS=YES

538ed47

required external libraries will be pulled in from the <>/pvxs/<TOP>/cfg

854eea8

add comments and instructions for enabling TLS support

3e63448

ernestow closed this Jun 14, 2025

ernestow reopened this Jun 14, 2025

ernestow marked this pull request as ready for review June 14, 2025 07:11

mdavidsaver marked this pull request as draft June 16, 2025 20:48

ernestow marked this pull request as ready for review June 25, 2025 07:52

mdavidsaver marked this pull request as draft June 25, 2025 16:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Building P4P against PVXS+TLS #177

Building P4P against PVXS+TLS #177

Uh oh!

ernestow commented Jun 11, 2025 •

edited by mdavidsaver

Loading

Uh oh!

mdavidsaver commented Jun 11, 2025

Uh oh!

mdavidsaver commented Jun 11, 2025

Uh oh!

ernestow commented Jun 12, 2025

Uh oh!

ernestow commented Jun 12, 2025

Uh oh!

ernestow commented Jun 14, 2025

Uh oh!

ernestow commented Jun 14, 2025

Uh oh!

ernestow commented Jun 14, 2025

Uh oh!

mdavidsaver commented Jun 16, 2025

Uh oh!

mdavidsaver commented Jun 16, 2025

Uh oh!

ernestow commented Jun 16, 2025

Uh oh!

ernestow commented Jun 16, 2025

Uh oh!

mdavidsaver commented Jun 23, 2025

Uh oh!

ernestow commented Jun 23, 2025 via email

Uh oh!

ernestow commented Jun 25, 2025

Uh oh!

mdavidsaver commented Jun 25, 2025

Uh oh!

Uh oh!

Building P4P against PVXS+TLS #177

Are you sure you want to change the base?

Building P4P against PVXS+TLS #177

Uh oh!

Conversation

ernestow commented Jun 11, 2025 • edited by mdavidsaver Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mdavidsaver commented Jun 11, 2025

Uh oh!

mdavidsaver commented Jun 11, 2025

Uh oh!

ernestow commented Jun 12, 2025

Add OpenSSL library paths for specific targets

Uh oh!

ernestow commented Jun 12, 2025

Uh oh!

ernestow commented Jun 14, 2025

Uh oh!

ernestow commented Jun 14, 2025

Uh oh!

ernestow commented Jun 14, 2025

Uh oh!

mdavidsaver commented Jun 16, 2025

Uh oh!

mdavidsaver commented Jun 16, 2025

Uh oh!

ernestow commented Jun 16, 2025

Uh oh!

ernestow commented Jun 16, 2025

Uh oh!

mdavidsaver commented Jun 23, 2025

Uh oh!

ernestow commented Jun 23, 2025 via email

Uh oh!

ernestow commented Jun 25, 2025

Uh oh!

mdavidsaver commented Jun 25, 2025

Uh oh!

Uh oh!

ernestow commented Jun 11, 2025 •

edited by mdavidsaver

Loading