Open System Services (OSS) Update and OSS Developer's...

©2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice

Marc Fuentes and Milind Dattawadkar

Open System Services (OSS) Update and OSS Developer's Tips

AGENDA

2

– Recent enhancements

• AF_UNIX Release 2

• OSS Limits Lifting

• POSIX User Threads

– Migration considerations

– Upcoming enhancements

• 64-bit OSS Processes

• Additional OSS Limits Lifting

• FLOSS and Open Source Utilities

AF_UNIX Release 2

3

– Overview

• Completely re-architectured implementation of AF_UNIX sockets on OSS

• Many fewer “moving parts”

• API compatibility with AF_UNIX Release 1 (the earlier implementation)

• AF_UNIX Release 2 totally replaces AF_UNIX Release 1

– Benefits

• Increased reliability and robustness

• Increased performance

– Availability

• AF_UNIX Release 2 is installed automatically when H06.16/J06.05 or later SUT is

installed.

• $ZPMON.#ZPLS remains in the OSS Monitor’s Database for fall-back to earlier versions

3

OSS Limits Lifting

4

– Overview

• More OSS file opens (64,000 opens per CPU)

− 48,000 disk opens; 16,000 sockets; 16,000 pipes

• More memory for file IO operations (> 100 MB per CPU)

− 60 MB disk cache; 32 MB sockets; 32 MB pipes

• Efficient memory allocation for file IO operations

• Resource usage EMS events

− Significant state changes and Failures

– Benefits

• Increased scalability

• Efficient use of memory

• Enhanced manageability

– Availability

• Automatically present and enabled when H06.18/J06.07 or later SUT is installed

4

POSIX User Threads

5

– Overview

• Thread-aware version of NSK public DLLs allow multi-threaded programs to safely execute

non-blocking library I/O functions and functions that share context.

• Stack overflow detection provides protected memory: process signal stack, and protected

thread stacks. It allows an OSS process to enable a signal handler to catch stack

overflow traps (SIGSTK).

– Benefits

• Improved performance for threaded applications

• Improved robustness in applications (Stack overflow detection)

• Improved POSIX standard compliance

– Availability

• Available when H06.21/J06.10 or later SUT is installed

• Applications need to be rebuilt with T1280, the POSIX User Thread (PUT) library

• Certain applications may need modifications

5

Relesase 1 (compatibility) and Release 2 (portability) mode behavioral differences (1 of 3)

AF_UNIX Release 2 features

6

– Release 1: getpeername() returns the relative pathname of the peer socket when bind() was given a relative pathname.

– Release 2: getpeername() returns the fully-qualified equivalent of the relative pathname of the peer socket when bind() was given a relative

pathname.

– Release 1 Datagram sockets: getpeername() returns ENOENT when the

peer socket file has been renamed

– Release 2 Datagram sockets: getpeername() returns the fully qualified name of the socket file given to connect()

– Release 1 Datagram sockets: getpeername() returns ENOENT when the

peer socket file has been unlinked

– Release 2 Datagram sockets: getpeername() returns the fully qualified name of the socket file given to connect()

Release 1 (compatibility) and Release 2 (portability) mode differences (2 of 3)


7

– Release 1 receivefrom()/receivemsg(): When the file to which a sending datagram socket is bound is removed and one of the send family of functions is called, the receiving client’s call to recvfrom() or recvmsg() returns a null address (all fields in the address are zero).

– Release 2 receivefrom()/receivemsg(): When the file to which a sending datagram socket is bound is removed and one of the send family of functions is called, the receiving client’s call to recvfrom() or recvmsg() returns the fully-qualified form of the address to which the sending socket was originally bound.

– Release 1 connect(): If listen() is called with a backlog <= 0, subsequent attempts to connect to the socket are rejected with ECONNREFUSED unless there is a pending accept() call on the socket.

– Release 2 connect(): If listen() is called with a backlog <= 0, subsequent attempts to connect to the socket block until there is a corresponding accept() call on the socket.

Release 1 (compatibility) and Release 2 (portability) mode differences (3 of 3)


8

– Release 1 getsockname(): When a bound socket file is unlinked or

renamed, a subsequent call to getsockname() fails with ENOENT.

– Release 2 getsockname(): When a bound socket file is unlinked or

renamed, a subsequent call to getsockname() still returns the fully

qualified name.

Specifying Release 1 (compatibility mode) or Release 2 (portability mode) behavior


9

– Processes can use both Release 1 and Release 2 sockets concurrently

– Sockets created in one mode cannot interact with sockets created in a different mode (connect() fails with ECONNREFUSED, sendto()/sendmsg() fails with EPERM)

– The default mode is Release 1 (compatibility mode)

– Changing the mode only affects the calling process and child processes, it does not change the mode for all present/future processes on the system

– The =_AFUNIX_PROCESS_NAME File System Define is used to set the default mode for processes that inherit the define

– Processes can call socket_transport_name_set() to set the mode

for the next socket the process creates.

Operations and usage

AF_UNIX Release 2 Operations and Management

10

– LS2 processes may be stopped for replacement.

• SCF STOP SERVER $ZPMON.#ZLSnn

• If there are AF_UNIX sockets in the CPU, the operation fails:

• 1-> assume process $zpmon

• 2-> stop server #zls00

• OSS E00030 Failed to stop server $ZLS00

• 3->

Selecting Release 1 behavior (compatibility mode)


11

– TACL (note that $ZPLS is not an actual process name under AF_UNIX Release 2)

• ADD DEFINE =_AFUNIX_PROCESS_NAME, CLASS MAP, FILE $ZPLS

– OSS shell

• add_define =_AFUNIX_PROCESS_NAME class=MAP file=\$ZPLS

– Programmatically

• if (DEFINESETATTR(“CLASS<blank-padded-for11-bytes>”,”MAP”,3)

exit(-1);

if(DEFINESETATTR(“FILE<blank-paded-for-12-bytes”,”$ZPLS”,5)

exit(-1);

if(2051 == DEFINEADD(“=_AFUNIX_PROCESS_NAME<blank-padded-for3-bytes>”,1))

if(DEFINEADD(“=_AF_UNIX_PROCESS_NAME<blank-padded-for-3-bytes>”,0)

exit(-1);

– Dynamically

• #define _XOPEN_SOURCE_EXTENDED 1

#include <sys/socket.h>

If(socket_transport_name_set(AF_UNIX, “$ZPLS”))

exit(-1);

Selecting Release 2 behavior (portability mode)


12

– TACL (note that $ZAFN2 is not an actual process name under AF_UNIX Release 2)

• ADD DEFINE =_AFUNIX_PROCESS_NAME, CLASS MAP, FILE $ZAFN2

– OSS shell

• add_define =_AFUNIX_PROCESS_NAME class=MAP file=\$ZAFN2

– Programmatically

• if(DEFINESETATTR(“CLASS<blank-padded-for11-bytes>”,”MAP”,3)

exit(-1);

if(DEFINESETATTR(“FILE<blank-paded-for-12-bytes”,”$ZAFN2”,6)

exit(-1);

if(2051 == DEFINEADD(“=_AFUNIX_PROCESS_NAME<blank-padded-for3-bytes>”,1))

if(DEFINEADD(“=_AF_UNIX_PROCESS_NAME<blank-padded-for-3-bytes>”,0)

exit(-1);

– Dynamically

• #define _XOPEN_SOURCE_EXTENDED 1

#include <sys/socket.h>

if(socket_transportname_set(AF_UNIX, “$ZAFN2”))

exit(-1);

New EMS events

OSS Limits Lifting

13

– Resources monitored

• POB open table, PXS buffers, socket open table, socket buffers, pipe open table, pipe

buffers, disk opens, disk cache buffers

– Event types

• Limit warning (When 85% of the resource has been allocated)

• Limit error (When 100% of the resource has been allocated)

• Status normal (When the resource usage falls to or below 65%)

• Allocation error (When the resource cannot be allocated for some other reason)

New EMS events – Limit warning event example

OSS Limits Lifting

14

2009-01-28 20:00:18 \YOSQA26.$ZPP01 TANDEM.OSS.H04 000041

Resource PIPEFIFO BUFS is approaching the resource limit.

Resource limit: 8388608

Warning threshold: 85

Current percentage: 85

Current usage: 7135232

Peak percentage: 100

Peak usage: 8404992

Peak time: 28JAN09 18:55

Reset time: 28JAN09 16:05

Safe threshold: 65

New EMS events – Limit error event example

OSS Limits Lifting

15

2009-01-28 20:00:33 \YOSQA26.$ZPP01 *TANDEM.OSS.H04 000040

Limit error for PIPEFIFO BUFS resource.




OSS PID: 65537

Program file: \YOSQA26.$SYSTEM.SYS52.OSSPS

Attempted Allocation Amount: 8192

New EMS events – Status normal event example

OSS Limits Lifting

16

2009-01-28 20:02:08 \YOSQA26.$ZPP01 TANDEM.OSS.H04 000042

Status returned to normal for PIPEFIFO BUFS resource.


Warning threshold: 85

Safe threshold: 65

Status threshold: 0




Peak usage: 8404992



New EMS events – Allocation error event example

OSS Limits Lifting

17

2009-01-28 19:49:33 \YOSQA26.$ZPP01 *TANDEM.OSS.H04 000039

Allocation error for PIPEFIFO BUFS resource.



Current usage: 0


Peak usage: 8404992



OSS PID: 65537

Program file: \YOSQA26.$SYSTEM.SYS52.OSSPS

Attempted Allocation Amount: 8192

Migrating from the SPT (T1248) library to the PUT library (T1280) –Compiler defines

POSIX User Threads

18

– Include pthread.h in /usr/include instead of spthread.h

• #include <pthread.h>

– Define _PUT_MODEL_

• This define is required for threaded applications using the PUT library. It may be specified

in the source code as a #define or on the compiler command line with the –D switch.

– Faster select() algorithm

• Define _PUT_SELECT_SINGLE_ to use a version of select() that is faster. This is

equivalent to SPT_SELECT_SINGLE in the SPT library

Migrating from the SPT library to the PUT library – Environment variables

POSIX User Threads

19

– PUT_THREAD_AWARE_REGULAR_IO_DISABLE

• Disables thread-aware regular I/O behavior. Thread-aware APIs ignore the O_NONBLOCK

flag for regular files. Setting this environment variable before running an application that

used the PUT library causes thread-aware APIs to have process blocking behavior. See

the Open System Services Programmer’s Guide for details.

– PUT_PROTECTED_STACK_DISABLE

• Setting this environment variable causes the thread stack to be allocated from the

process’ heap instead of being allocated on a separate protected stack

Migrating from the SPT library to the PUT library

20

– Mixing the SPT library and the PUT library in the same process is not allowed, attempts to do so fail with runtime error 019

– pthread_attr_default, pthread_mutexattr_default, pthread_condattr_default do not exist as global varables in the

PUT library

– The PUT library functions return -1 and set errno rather than returning the errno value.

– unistd.h has a set of macros that influence HP’s thread

implementation. See the Open System Services Programmer’s Guide for details.

– Thread-aware signals are always enabled in the PUT library.

– PUT-based applications have to statically link with the PUT library.

POSIX User Threads migration considerations

Signal handling topics (1 of 3)


21

– PUT model applications cannot call sigaltstack() to establish a signal stack. They can call sigaltstack() to find the information about the

signal stack.

– The Open System Services Programmer’s Guide lists a set of functions that are not async-signal safe. These functions should not be used in signal handlers (this is similar to the SPT library).

– Job control signals and their corresponding actions are not supported (this is similar to the SPT library).

– The Real-time Signals Extension option is not supported (this is similar to the SPT library).

– signal() is deprecated, use sigaction() instead

– See the Open System Services Programmer’s Guide for a list of APIs that support thread-aware signal handling


22

Similar to the SPT library

– Some signals affect the entire process. See the Open System Services Programmer’s Guide for details

– If a signal is delivered to a thread waiting on a condition variable, then upon return from the signal handler, the thread will resume waiting for the condition variable as if it were not interrupted.

– Do not use the raise() API for an exception, and do not use the longjmp() API to exit a synchronous signal handler. Simply returning from a signal handler is safe and is the recommended practice.

– Use exception handlers instead of the setjmp() and longjmp() APIs for threaded OSS applications.

– Do not use the option of the sigsetjmp() and siglongjmp() APIs that saves and restores the signal mask. Doing so can cause signals enabled by another thread to be masked.



23

– Do not deliver signals like SIGALRM and SIGCHLD using the kill

command or the raise() function because these do not provide

enough information about the process IDs for the parent and child

processes. Instead, use the PUT Model library version of the alarm()

API to raise alarms.


APIs in the SPT library that do not exist in the PUT library

24

– The PUT model library does not provide analogs to spt_*x() or

spt_*z() APIs, such as spt_readx() or spt_readz(), whose

purpose is to be non-blocking alternatives to standard system library

functions. Instead, use the standard API names.

– The PUT model library does not provide analogs to spt_*() functions,

such as spt_fprintf(), whose purpose is to be non-blocking

alternatives to the standard C run-time library functions. Instead, use the

standard API names. The C run-time library functions are thread-aware

if called from a PUT-based application.


Pitfalls

25

– See the Open System Services Programmer’s Guide for a list of thread-

aware APIs, a list of non-thread-safe APIs, non-thread-safe cancellation

points, and a list of APIs that are potential cancellation points

– Do not mix modules compiled to use the PUT library with non thread-

aware modules when the modules perform I/O to the same files

– Look out for warnings of the form: function “<function-name>" declared implicitly,

reports that no explicit definition of <function-name>() appeared and

therefore the compiler assumed defaults for all function attributes.

Consequently, calls to <function-name>() are not aliased to the non-

blocking variant and cause an unneeded and unwanted suspension of

the entire process.


Post-POSIX User Thread

Thread-Aware I/O Availability

called directly from:

Thread-aware system I/O on non-disk file

Thread-aware system I/O on disk file

Thread-aware C I/O on non-disk file

Thread-aware C I/O on disk file

Thread-aware C++ I/O streaming

SPT-based application √ √ √ Х Х

Public DLL linked to SPT-based applications

Х Х Х Х Х

called directly from:

Thread-aware system I/O on non-disk file

Thread-aware system I/O on disk file

Thread-aware C I/O on non-disk file

Thread-aware C I/O on disk file

Thread-aware C++ I/O streaming

Thread-aware public DLL functions

SPT-based application √ √ √ Х Х Х

Public DLL linked to SPT-based applications

Х Х Х Х Х Х

PUT-based application √ √ √ √ √ √

Public DLL linked to PUT-based applications

√ √ √ √ √ √

Pre-POSIX User Thread

26

64 Bit OSS Processes

27

– Overview for C/C++ programs

• Allow OSS programs access to a much larger virtual address space for data through

support of 64-bit addressing.

− SQL/MX Access from processes using 64-bit addressing.

• Run-time heap in 64-bit addressable virtual memory

− Default limit is 12GB

− NonStop OS limit is .5TB for heap and 64-bit flat segments.

− Practical limit determined by the amount of RAM and disk space for swap files.

– Benefits


Future product plans, dates, and functionality are subject to change without notice.

27

Additional OSS Limits Lifting

28

– Overview

• More OSS processes per system

− 128,000 simultaneous OSS processes on a 16-processor system

• More OSS file opens (128,000 opens per CPU)

− 96,000 disk opens; 32,000 sockets; 32,000 pipes; 32,000 ttys

• More memory for file IO operations

− 120 MB disk cache; 64 MB sockets; 32 MB pipes

• Resource monitoring capabilities with SCF commands

• Resource monitoring capabilities with Measure

– Benefits


• Enhanced resource monitoring

Future product plans, dates, and functionality are subject to change without notice.

28

FLOSS and Open Source Utilities

29

– Overview

• FLOSS - enables porting of open source tools to the NonStop platform

− At present, FLOSS version 0.9 is available at the Connect website

(http://ituglib.connect-community.org/floss-0.9-src.tar.gz). This will be made

available as a standard product on the RVU

• Provides select Open Source Tools on the NonStop platform

− FLOSS will be officially supported

− Tools made available on a business case by case basis

− Samba

• Update tools on the Connect website (BASH, Archival tools, Core utilities, etc.)

• Provide RPM and RPM based packages on the Connect website to ease installation and

dependency management

– Benefits

• Enable portability of open source packages

• Updated tools

Future product plans, dates, and functionality are subject to change without notice.29

HP Integrity NonStop modernization for greater infrastructure convergence

ST EPup the momentum of your technology evolution.

Date post:	12-Mar-2021
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Open System Services (OSS) Update and OSS Developer's...

Documents