+ All Categories
Home > Documents > CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration

Date post: 12-Jan-2016
Category:
Upload: ingo
View: 51 times
Download: 0 times
Share this document with a friend
Description:
CIT 470: Advanced Network and System Administration. Logging. Topics. System logs Logging policies Finding logs Syslog Syslog servers Log monitoring. System Logs. Logs record status and error conditions. Where do log messages come from? Kernel Accounting system System services - PowerPoint PPT Presentation
Popular Tags:
27
CIT 470: Advanced Network and System Administration Slide #1 CIT 470: Advanced Network and System Administration Logging
Transcript
Page 1: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #1

CIT 470: Advanced Network and System Administration

Logging

Page 2: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #2

Topics

1. System logs

2. Logging policies

3. Finding logs

4. Syslog

5. Syslog servers

6. Log monitoring

Page 3: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #3

System Logs

• Logs record status and error conditions.

• Where do log messages come from?– Kernel– Accounting system– System services

• Logging methods:– Service records own logs (apache, cron).– Service uses syslog service to manage logs.

Page 4: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #4

Logging Policies

1. Throw away log data.

2. Save for a while, then throw away.

3. Rotate log files

4. Archive log files

Page 5: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #5

How to choose a logging policy?

1. Are there any data retention requirements?

2. How much disk space do you have?

3. How quickly do you need to retrieve logs?

4. Could you find the source of a security issue with the logs you keep?

Page 6: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #6

Throwing Away

• Not recommended.

• Leaves you unaware of:– Software and hardware problems– Security incidents

• It may take time to detect an incident.– Keep logs for at least a month or two.

Page 7: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #7

Rotation

• Keep backup files for each day/weeklogfile

logfile.1

logfile.2

logfile.3

• Rename files each day/week to move old ones back in list.– Compress rotated logs to save disk space.– Remove/archive logs that are X days old.

Page 8: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #8

Rotation

#!/bin/sh

cd /var/log

mv logfile.2 logfile.3

mv logfile.1 logfile.2

mv logfile logfile.1

cp /dev/null logfile

chmod 600 logfile

Page 9: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #9

logrotate

• Program to handle log rotation.– Run via /etc/cron.daily.– Configured via /etc/logrotate.conf.

• Options– How often to rotate– How long to keep logs– Compression or not– Log file permissions– Pre- and post-rotate scripts

Page 10: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #10

logrotate.conf# rotate log files weeklyweekly# keep 4 weeks worth of backlogsrotate 4# create new (empty) log files after rotating oldcreate# uncomment if you want your log files compressed#compress# RPM packages drop log rotation information intoinclude /etc/logrotate.d# no packages own wtmp -- we'll rotate them here/var/log/wtmp { monthly create 0664 root utmp rotate 1}

Page 11: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #11

Archiving Logs

• Store logs to archival media (tape.)– Archive after X days/weeks.– Should be part of regular backup plan.– May want to save logs for all hosts together.

Page 12: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #12

Finding Logs

• Most logs are stored under– /var/log– /var/adm

• To find other logs, read startup scripts– /etc/init.d/*– and manuals for services started by scripts.

Page 13: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #13

Log file Program Contents

messages syslog Various program/kernel logs.

syslog syslog Various program/kernel logs.

auth.log su, ssh, login Authorization fail/success.

lastlog login, xdm Logins, commands.

wtmp login Login accounting data.

acct/pacct kernel UNIX process accounting.

Xorg.log X-Windows X-Windows failures/info.

Finding Logs

Page 14: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #14

Syslog

Comprehensive logging system.Frees programmers from managing log files.

Gives sysadmins control over log management.

Sorts messages bySources

Importance

Routes messages to destinationsFiles

Network

Terminals

Page 15: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #15

Syslog Components

SyslogDaemon that does actual logging.

Additional daemon, klog, gets kernel messages.

openlog, syslog, closelogC library routines to submit logs to syslog.

loggerUser-level program to submit logs to syslog.

Can use from shell scripts.

Page 16: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #16

Example Syslog MessagesFeb 11 10:17:01 localhost /USR/SBIN/CRON[1971]: (root) CMD ( run-

parts --report /etc/cron.hourly)Feb 11 10:37:22 localhost -- MARK --Feb 11 10:51:11 localhost dhclient: DHCPREQUEST on eth1 to

192.168.1.1 port 67Feb 11 10:51:11 localhost dhclient: DHCPACK from 10.42.1.1Feb 11 10:51:11 localhost dhclient: bound to 10.42.1.55 -- renewal in

35330 seconds.Feb 11 14:37:22 localhost -- MARK --Feb 11 14:44:21 localhost mysqld[7340]: 060211 14:44:21

/usr/sbin/mysqld: Normal shutdownFeb 12 04:46:42 localhost sshd[29093]: Address 218.38.30.101 maps to

ns.thundernet.co.kr, but this does not map back to the address - POSSIBLE BREAKIN ATTEMPT!

Feb 12 04:46:44 localhost sshd[29097]: Invalid user matt from ::ffff:218.38.30.101

Page 17: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #17

Configuring Syslog

Configured in /etc/syslog.confFormat: selector <Tab> action

Ex: mail.info /var/log/mail.log

Selector componentsSource (facility)

List of facilities separated by commas or *.

Importance (level)

Can be none or *

Page 18: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #18

/etc/syslog.conf# Log anything (except mail) of level info or higher.# Don't log private authentication messages!*.info;mail.none;authpriv.none;cron.none /var/log/messages

# The authpriv file has restricted access.authpriv.* /var/log/secure

# Log all the mail messages in one place.mail.* /var/log/maillog

# Log cron stuffcron.* /var/log/cron

# Everybody gets emergency messages*.emerg *

# Save news errors of level crit and higher in a special file.uucp,news.crit /var/log/spooler

# Save boot messages also to boot.loglocal7.* /var/log/boot.log

Page 19: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #19

Syslog FacilitiesFacility Used By

kern The kernel

user User processes (default)

mail Mail servers and related software.

daemon System daemons (except mail, cron)

auth Security and authorization-related commands.

lpr Print server and related commands.

cron Cron daemon.

local0-7 Eight local levels for other programs.

Page 20: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #20

Syslog LevelsLevel Meaning

emerg Panic situations (hardware failure, crash)

alert Urgent situations

crit Critical situations

err Non-critical errors.

warning Warnings.

notice Might merit investigation.

info Informational messages.

debug Debugging (typically enabled temporarily.)

Page 21: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #21

Syslog ActionsAction Meaning

filename Write message to file on local machine.

@hostname Send message to syslogd on hostname.

@ip Send message to syslogd at IP address.

user1,user2 Write message to user screen if logged in.

* Write message to all logged-in users.

Page 22: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #22

Testing Syslogstu> for i in {debug,info,notice,warning,err,crit,alert,emerg}> do> logger -p daemon.$i "Test message for daemon, level $i"> donestu> tail /var/log/daemon.logFeb 11 15:57:00 localhost stu: Test message for daemon, level debugFeb 11 15:57:00 localhost stu: Test message for daemon, level infoFeb 11 15:57:00 localhost stu: Test message for daemon, level noticeFeb 11 15:57:00 localhost stu: Test message for daemon, level warningFeb 11 15:57:00 localhost stu: Test message for daemon, level errFeb 11 15:57:00 localhost stu: Test message for daemon, level critFeb 11 15:57:00 localhost stu: Test message for daemon, level alertFeb 11 15:57:00 localhost stu: Test message for daemon, level emerg

Page 23: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #23

Syslog Variants

Some use m4 macrosauth.notice ifdef(‘LOGHOST’, ‘/var/log/authlog’, ‘@loghost’)

Red Hat Linux variantsAllows spaces as separators.

New operators: = (this priority only)Ex: mail.=info

New operators: ! (except this pri and higher)Ex: mail.info,mail.!err

Page 24: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #24

Syslog NG

Free drop-in replacement for syslog.More configurable

Save logs to templated location (auto-rotates.)Filter logs based on program, time, message, etc.Message format customization.Allows easy logging to remote database.

Improved networkingTCP support as well as UDP.

Improved securityDoesn’t trust hostnames in remote messages.TCP transmission permits encrypted tunneling (stunnel.)

Page 25: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #25

Log Servers

Collect all syslog data on one server.Allows logging to scale to large networks.

Logs can be correlated across machines.

Security-sensitive logs not on compromised host.

Routers and diskless-hosts must log to a server.

Need two syslog.conf filesClient: sends all logs across network to server.

Server: saves logs to database or local files.

Page 26: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #26

Log Monitoring

Too much data for a human to process.Logs arrive 24x7 too.

Use an automatic monitoring programTriggers on patterns found in log.

Examples: logwatch, swatch

# 3ware logswatchfor /(?i)3w-xxxx.+no longer fault tolerant/mail=root,subject=LW warn: disk 3ware RAID not fault tolerantthrottle 1:00:00,use=regex

Page 27: CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Slide #27

References1. Michael Bower, Building Secure Servers with Linux,

O’Reilly, 2005.2. Aeleen Frisch, Essential System Administration, 3rd

edition, O’Reilly, 2002.3. Jeremy Mate, “Log Analysis with Swatch,”

http://sial.org/howto/logging/swatch/, 2005.4. Jeremy Mate, “Logging with syslog-ng,”

http://sial.org/howto/logging/syslog-ng/, 2005.5. Evi Nemeth et al, UNIX System Administration

Handbook, 3rd edition, Prentice Hall, 2001.6. Shelley Powers et. al., UNIX Power Tools, 3rd edition,

O’Reilly, 2002.7. Syslog-ng FAQ,

http://www.campin.net/syslog-ng/faq.html.


Recommended