1. LinuxKernelNetworkingRamiRosenramirose@gmail.comHaifux,August2007

2. Disclaimer Everything in this lecture shall not, under any circumstances, hold any legal liability whatsoever. Any usage of the data and information in this document shall be solely on the responsibility of the user. This lecture is not given on behalf of any company or organization. 3. Warning Thislecturewilldealwithdesignfunctionaldescriptionsidebysidewithmanyimplementationdetails;someknowledgeofCispreferred. 4. General TheLinuxnetworkingkernelcode(includingnetworkdevice drivers)isalargepartoftheLinuxkernelcode. Scope:Wewillnotdealwithwireless,IPv6,andmulticasting. Alsonotwithuserspaceroutingdaemons/apps,andwithsecurityattacks(likeDoS,spoofing,etc.). Understandingapacketwalkthroughinthekernelisakeyto understandingkernelnetworking.UnderstandingitisamustifwewanttounderstandNetfilterorIPSecinternals,andmore. Thereisa10pagesLinuxkernelnetworkingwalkthrouhdocument 5. GeneralContd. Thoughitdealswith2.4.20Linuxkernel,mostofitisrelevant. Thislecturewillconcentrateonthiswalkthrough(designand implementationdetails). Referencestocodeinthislecturearebasedonlinux2.6.23rc2. Therewassomeseriouscleanupin2.6.23 6. Hierarchyofnetworkinglayers Thelayersthatwewilldealwith(basedonthe7layersmodel)are: TransportLayer(L4)(udp,tcp...) NetworkLayer(L3)(ip)LinkLayer(L2)(ethernet) 7. NetworkingDataStructures Thetwomostimportantstructuresoflinuxkernelnetworklayer are: sk_buff(definedininclude/linux/skbuff.h) netdevice(definedininclude/linux/netdevice.h) Itisbettertoknowabitaboutthembeforedelvingintothe walkthroughcode. 8. SK_BUFF sk_buffrepresentsdataandheaders. sk_buffAPI(examples) sk_buffallocationisdonewithalloc_skb()ordev_alloc_skb();driversusedev_alloc_skb();.(freebykfree_skb()anddev_kfree_skb(). unsignedchar*data:pointstothecurrentheader. skb_pull(intlen)removesdatafromthestartofabufferby advancingdatatodata+lenandbydecreasinglen. Almostalwayssk_buffinstancesappearasskbinthekernel code. 9. SK_BUFFcontd sk_buffincludes3unions;eachcorrespondstoakernelnetwork layer: transport_header(previouslycalledh)forlayer4,thetransport layer(canincludetcpheaderorudpheaderoricmpheader,and more) network_header(previouslycallednh)forlayer3,thenetwork layer(canincludeipheaderoripv6headerorarpheader). mac_header(previouslycalledmac)forlayer2,thelinklayer. skb_network_header(skb),skb_transport_header(skb)and skb_mac_header(skb)returnpointertotheheader. 10. SK_BUFFcontd. structdst_entry*dsttherouteforthissk_buff;thisrouteis determinedbytheroutingsubsystem. Ithas2importantfunctionpointers: int(*input)(structsk_buff*); int (*output)(structsk_buff*); input()canbeassignedtooneofthefollowing:ip_local_deliver, ip_forward,ip_mr_input,ip_errorordst_discard_in. output()canbeassignedtooneofthefollowing:ip_output, ip_mc_output,ip_rt_bug,ordst_discard_out. 11. SK_BUFFcontd. Intheusualcase,thereisonlyonedst_entryforeveryskb. WhenusingIPSec,thereisalinkedlistofdst_entriesandonlythe lastoneisforrouting;allotherdst_entriesareforIPSec transformers;theseotherdst_entrieshavetheDST_NOHASH flagset. tstamp(oftypektime_t):timestampofreceivingthepacket. net_enable_timestamp()mustbecalledinordertogetvalues. 12. net_device net_devicerepresentsanetworkinterfacecard. Therearecaseswhenweworkwithvirtualdevices. Forexample,bonding(settingthesameIPfortwoormoreNICs,forloadbalancingandforhighavailability.) Manytimesthisisimplementedusingtheprivatedataofthedevice(thevoid*privmemberofnet_device); InOpenSolaristhereisaspecialpseudodrivercalledvnicwhichenablesbandwidthallocation(projectCrossBow). Importantmembers: 13. net_devicecontd unsignedintmtuMaximumTransmissionUnit:themaximum sizeofframethedevicecanhandle. Eachprotocolhasmtuofitsown;thedefaultis1500forEthernet. youcanchangethemtuwithifconfig;forexample,likethis: ifconfigeth0mtu1400 Youcannotofcourse,changeittovalueshigherthan1500on10Mb/snetwork: ifconfigeth0mtu1501willgive: SIOCSIFMTU:Invalidargument 14. net_devicecontd unsignedintflags(whichyouseeorsetusingifconfigutility): forexample,RUNNINGorNOARP. unsignedchardev_addr[MAX_ADDR_LEN]:theMACaddress ofthedevice(6bytes). int (*hard_start_xmit)(structsk_buff*skb,structnet_device*dev); apointertothedevicetransmitmethod. int promiscuity;(acounterofthetimesaNICistoldtosetto workinpromiscuousmode;usedtoenablemorethanonesniffingclient.) 15. net_devicecontd YouarelikelytoencountermacrosstartingwithIN_DEVlike:IN_DEV_FORWARD()orIN_DEV_RX_REDIRECTS().Howarethe relatedtonet_device?Howarethesemacrosimplemented? void*ip_ptr:IPv4specificdata.Thispointerisassignedtoa pointertoin_deviceininetdev_init()(net/ipv4/devinet.c) 16. net_deviceContd. structin_devicehaveamembernamedcnf(instanceof ipv4_devconf).Setting/proc/sys/net/ipv4/conf/all/forwardingeventuallysetstheforwardingmemberofin_deviceto1.Thesameistruetoaccept_redirectsandsend_redirects;botharealsomembersofcnf(ipv4_devconf). Inmostdistros,/proc/sys/net/ipv4/conf/all/forwarding=0 ButprobablythisisnotsoonyourADSLrouter. 17. networkinterfacedrivers MostofthenicsarePCIdevices;therearealsosomeUSB networkdevices. ThedriversfornetworkPCIdevicesusethegenericPCIcalls,likepci_register_driver()andpci_enable_device(). FormoreinfoonnicdrivesseethearticleWritingNetwork DeviceDriverforLinux(linkno.9inlinks)andchap17inldd3. TherearetwomodesinwhichaNICcanreceiveapacket. Thetraditionalwayisinterruptdriven:eachreceivedpacketis anasynchronouseventwhichcausesaninterrupt. 18. NAPI NAPI(newAPI). TheNICworksinpollingmode. Inorderthatthenicwillworkinpollingmodeitshouldbebuiltwithaproperflag. Mostofthenewdriverssupportthisfeature. WhenworkingwithNAPIandwhenthereisaveryhighload,packetsarelost;butthisoccursbeforetheyarefedintothe networkstack.(inthenonNAPIdrivertheypassintothestack) inSolaris,pollingisbuiltintothekernel(noneedtobuild 19. UserSpaceTools iputils(includingping,arping,andmore) nettools(ifconfig,netstat,,route,arpandmore) IPROUTE2(ipcommandwithmanyoptions) UsesrtnetlinkAPI. Hasmuchwiderfunctionalities;forexample,youcancreatetunnelswithipcommand. Note:noneedfornflagwhenusingIPROUTE2(becauseitdoesnotworkwithDNS). 20. RoutingSubsystem Theroutingtableandtheroutingcacheenableustofindthenet deviceandtheaddressofthehosttowhichapacketwillbesent. Readingentriesintheroutingtableisdonebycalling fib_lookup(conststructflowi*flp,structfib_result*res) FIBistheForwardingInformationBase. Therearetworoutingtablesbydefault:(nonPolicyRoutingcase) localFIBtable(ip_fib_local_table;ID255). mainFIBtable(ip_fib_main_table;ID254) See:include/net/ip_fib.h. 21. RoutingSubsystemcontd. Routescanbeaddedintothemainroutingtableinoneof3ways: Bysysadmincommand(routeadd/iproute). Byroutingdaemons. AsaresultofICMP(REDIRECT). Aroutingtableisimplementedbystructfib_table. 22. RoutingTables fib_lookup()firstsearchesthelocalFIBtable(ip_fib_local_table). Incaseitdoesnotfindanentry,itlooksinthemainFIBtable (ip_fib_main_table). Whyisitinthisorder? Thereisoneroutingcache,regardlessofhowmanyroutingtables thereare. YoucanseetheroutingcachebyrunningrouteC. Alternatively,youcanseeitby:cat/proc/net/rt_cache. con:thisway,theaddressesareinhexformat 23. RoutingCache Theroutingcacheisbuiltofrtableelements: structrtable(see:/include/net/route.h) {union{structdst_entry dst;}u;... } 24. RoutingCachecontd Thedst_entryistheprotocolindependentpart. Thus,forexample,wehaveadst_entrymember(alsocalleddst)inrt6_infoinipv6.(include/net/ip6_fib.h) ThekeyforalookupoperationintheroutingcacheisanIP address(whereasintheroutingtablethekeyisasubnet). Insertingelementsintotheroutingcacheby:rt_intern_hash() Thereisanalternatemechanismforroutecachelookup, calledfib_trie,whichisinsidethekerneltree (net/ipv4/fib_trie.c) 25. RoutingCachecontd Itisbasedonextendingthelookupkey. Youshouldset:CONFIG_IP_FIB_TRIE(=y) (insteadofCONFIG_IP_FIB_HASH) ByRobertOlssonetal(seelinks). 26. CreatingaRoutingCacheEntry Allocationofrtableinstance(rth)isdoneby:dst_alloc(). dst_alloc()infactcreatesandreturnsapointertodst_entryandwecastittortable(net/core/dst.c). Settinginputandoutputmethodsofdst: (rth>u.dst.inputandrth>u.dst.input) Settingtheflowimemberofdst(rth>fl) Nexttimethereisalookupinthecache,forexample, ip_route_input(),wewillcompareagainstrth>fl. 27. RoutingCacheContd. Agarbagecollectioncallwhichdeleteeligibleentriesfromtheroutingcache. Whichentriesarenoteligible? 28. PolicyRouting(multipletables) Genericroutingusesdestinationaddressbaseddecisions. Therearecaseswhenthedestinationaddressisnotthesole parametertodecidewhichroutetogive;PolicyRoutingcomesto enablethis. 29. PolicyRouting(multipletables)contd. Addingaroutingtable:byaddingalineto:/etc/iproute2/rt_tables. Forexample:addtheline252my_rt_table. Therecanbeupto255routingtables. Policyroutingshouldbeenabledwhenbuildingthekernel (CONFIG_IP_MULTIPLE_TABLESshouldbeset.) Exampleofaddingarouteinthistable: >iprouteadddefaultvia192.168.0.1tablemy_rt_table Showthetableby: iprouteshowtablemy_rt_table 30. PolicyRouting(multipletables)contd. Youcanaddaruletotheroutingpolicydatabase(RPDB)byipruleadd... Therulecanbebasedoninputinterface,TOS,fwmark(fromnetfilter). iprulelistshowallrules. 31. PolicyRouting:add/deletearuleexample ipruleaddtos0x04table252 Thiswillcausepacketswithtos=0x08(intheiphdr) toberoutedbylookingintothetableweadded(252) Sothedefaultgwforthesetypeofpacketswillbe 192.168.0.1 ipruleshowwillgive: 32765:fromalltosreliabilitylookupmy_rt_table ... 32. PolicyRouting:add/deletearuleexample Deletearule:ipruledeltos0x04table252 33. RoutingLookupHit ip_route_input()in:net/ipv4/route.cCachelookup Miss Deliverpacketby: ip_route_input_slow()fib_lookup()inHit ip_local_deliver() in:net/ipv4/route.c ip_fib_local_tableorip_forward() accordingtoresultMissfib_lookup()inHitip_fib_main_tableMiss Droppacket 34. RoutingTableDiagram fib_table structfn_zonestructfib_node fib_nodehlist_head tb_lookup() fz_hash fn_aliastb_insert()hlist_head fn_alias tb_delete() fn_key hlist_head fn_key...structfib_aliasstructfn_zonefz_divisorstructfn_zonehlist_headfa_info... 33... structfn_zone structfib_info fib_nh 35. RoutingTables Breakingthefib_tableintomultipledatastructuresgives flexibilityandenablesfinegrainedandhighlevelofsharing. Supposethatwe10routesto10differentnetworkshavethesamenexthopgw. Wecanhaveonefib_infowhichwillbesharedby10fib_aliases. fz_divisoristhenumberofbuckets 36. RoutingTablescontd Eachfib_nodeelementrepresentsauniquesubnet. Thefn_keymemberoffib_nodeisthesubnet(32bit) 37. RoutingTablescontd Supposethatadevicegoesdownorenabled. Weneedtodisable/enableallrouteswhichusethisdevice. Buthowcanweknowwhichroutesusethisdevice? Inordertoknowitefficiently,thereisthefib_info_devhash table. Thistableisindexedbythedeviceidentifier. Seefib_sync_down()andfib_sync_up()in net/ipv4/fib_semantics.c 38. RoutingTablelookupalgorithm LPM(LongestPrefixMatch)isthelookupalgorithm. Theroutewiththelongestnetmaskistheonechosen. Netmask0,whichistheshortestnetmask,isforthedefault gateway. Whathappenswhentherearemultipleentrieswithnetmask=0? fib_lookup()returnsthefirstentryitfindsinthefibtablewherenetmasklengthis0. 39. RoutingTablelookupcontd. Itmaybethatthisisnotthebestchoicedefaultgateway. Soincasethatnetmaskis0(prefixlenofthefib_resultreturned fromfib_lookis0)wecallfib_select_default(). fib_select_default()willselecttheroutewiththelowestpriority(metric)(bycomparingtofib_priorityvaluesofalldefault gateways). 40. Receivingapacket Whenworkingininterruptdrivenmodel,thenicregistersan interrupthandlerwiththeIRQwithwhichthedeviceworksby callingrequest_irq(). Thisinterrupthandlerwillbecalledwhenaframeisreceived Thesameinterrupthandlerwillbecalledwhentransmissionofa frameisfinishedandunderotherconditions.(dependsonthe NIC;sometimes,theinterrupthandlerwillbecalledwhenthereis someerror). 41. Receivingapacketcontd Typicallyinthehandler,weallocatesk_buffbycalling dev_alloc_skb();alsoeth_type_trans()iscalled;amongother thingsitadvancesthedatapointerofthesk_bufftopointtotheIP header;thisisdonebycallingskb_pull(skb,ETH_HLEN). See:net/ethernet/eth.c ETH_HLENis14,thesizeofethernetheader. 42. Receivingapacketcontd Thehandlerforreceivingapacketisip_rcv().(net/ipv4/ip_input.c) Handlerfortheprotocolsareregisteredatinitphase. Likewise,arp_rcv()isthehandlerforARPpackets. First,ip_rcv()performssomesanitychecks.Forexample: if(iph>ihlversion!=4)gotoinhdr_error; iphistheipheader;iph>ihlistheipheaderlength(4bits). Theipheadermustbeatleast20bytes. Itcanbeupto60bytes(whenweuseipoptions) 43. Receivingapacketcontd Thenitcallsip_rcv_finish(),by:NF_HOOK(PF_INET,NF_IP_PRE_ROUTING,skb,dev,NULL, ip_rcv_finish); Thisdivisionofmethodsintotwostages(wherethesecondhas thesamenamewiththesuffixfinishorslow,istypicalfor networkingkernelcode.) Inmanycasesthesecondmethodhasaslowsuffixinsteadof finish;thisusuallyhappenswhenthefirstmethodlooksinsome cacheandthesecondmethodperformsalookupinatable,which isslower. 44. Receivingapacketcontd ip_rcv_finish()implementation:if(skb>dst==NULL){interr=ip_route_input(skb,iph>daddr,iph>saddr,iph>tos,skb>dev);...}... returndst_input(skb); 45. Receivingapacketcontd ip_route_input():Firstperformsalookupintheroutingcachetoseeifthereisa match.Ifthereisnomatch(cachemiss),calls ip_route_input_slow()toperformalookupintheroutingtable. (Thislookupisdonebycallingfib_lookup()). fib_lookup(conststructflowi*flp,structfib_result*res)Theresultsarekeptinfib_result. ip_route_input()returns0uponsuccessfullookup.(alsowhen thereisacachemissbutasuccessfullookupintheroutingtable.) 46. ReceivingapacketcontdAccordingtotheresultsoffib_lookup(),weknowiftheframeisforlocaldeliveryorforforwardingortobedropped. Iftheframeisforlocaldelivery,wewillsettheinput()function pointeroftheroutetoip_local_deliver():rth>u.dst.input=ip_local_deliver; Iftheframeistobeforwarded,wewillsettheinput()function pointertoip_forward():rth>u.dst.input=ip_forward; 47. LocalDelivery Prototype:ip_local_deliver(structsk_buff*skb)(net/ipv4/ip_input.c).callsNF_HOOK(PF_INET,NF_IP_LOCAL_IN,skb,skb>dev, NULL,ip_local_deliver_finish); Deliversthepackettothehigherprotocollayersaccordingtoits type. 48. Forwarding Prototype: intip_forward(structsk_buff*skb) (net/ipv4/ip_forward.c) decreasesthettlintheipheader Ifthettlisdev,rt>u.dst.dev,ip_forward_finish); 49. ForwardingContd ip_forward_finish():sendsthepacketoutbycalling dst_output(skb). dst_output(skb)isjustawrapper,whichcallsskb>dst>output(skb).(seeinclude/net/dst.h) 50. SendingaPacket Handlingofsendingapacketisdoneby ip_route_output_key(). Weneedtoperformroutinglookupalsointhecaseoftransmission. Incaseofacachemiss,wecallsip_route_output_slow(),whichlooksintheroutingtable(bycallingfib_lookup(),as alsoisdoneinip_route_input_slow().) Ifthepacketisforaremotehost,wesetdst>outputto ip_output() 51. SendingaPacketcontd ip_output()willcallip_finish_output() ThisistheNF_IP_POST_ROUTINGpoint. ip_finish_output()willeventuallysendthepacketfroma neighborby: dst>neighbour>output(skb) arp_bind_neighbour()seestoitthattheL2addressofthenexthopwillbeknown.(net/ipv4/arp.c) 52. SendingaPacketContd. Ifthepacketisforthelocalmachine: dst>output=ip_output dst>input=ip_local_deliver ip_output()willsendthepacketontheloopbackdevice, Thenwewillgointoip_rcv()andip_rcv_finish(),butthistimedstisNOTnull;sowewillendinip_local_deliver(). See:net/ipv4/route.c 53. Multipathrouting Thisfeatureenablestheadministratortosetmultiplenext hopsforadestination. Toenablemultipathrouting, CONFIG_IP_ROUTE_MULTIPATHshouldbesetwhen buildingthekernel. Therewasalsoanoptionformultipathcaching:(bysetting CONFIG_IP_ROUTE_MULTIPATH_CACHED). Itwasexperimentalandremovedin2.6.23Seelinks(6). 54. 55. Netfilter Netfilteristhekernellayertosupportapplyingiptablesrultes. Itenables: Filtering Changingpackets(masquerading) ConnectionTracking 56. Netfilterruleexample Shortexample: Applyingthefollowingiptablesrule: iptablesAINPUTpudpdport9999jDROP ThisisNF_IP_LOCAL_INrule; Thepacketwillgoto: ip_rcv() andthen:ip_rcv_finish() Andthenip_local_deliver() 57. Netfilterruleexample(contd) butitwillNOTproceedtoip_local_deliver_finish()asinthe usualcase,withoutthisrule. Asaresultofapplyingthisruleitreachesnf_hook_slow() withverdict==NF_DROP(callsskb_free()tofreethepacket) See/net/netfilter/core.c. 58. ICMPredirectmessage ICMPprotocolisusedtonotifyaboutproblems. AREDIRECTmessageissentincasetheroute issuboptimal(inefficient). Thereareinfact4typesofREDIRECT Onlyoneisused: RedirectHost(ICMP_REDIR_HOST) SeeRFC1812(RequirementsforIPVersion4Routers). 59. ICMPredirectmessagecontd. TosupportsendingICMPredirects,themachineshouldbe configuredtosendredirectmessages. /proc/sys/net/ipv4/conf/all/send_redirectsshouldbe1. Inorderthattheothersidewillreceiveredirects,weshould set /proc/sys/net/ipv4/conf/all/accept_redirectsto1. 60. ICMPredirectmessagecontd. Example: Addasuboptimalrouteon192.168.0.31: routeaddnet192.168.0.10netmask255.255.255.255gw 192.168.0.121 Runningnowrouteon192.168.0.31willshowanewentry:DestinationGatewayGenmaskFlagsMetricRefUseIface192.168.0.10192.168.0.121255.255.255.255UGH000eth0 61. ICMPredirectmessagecontd. Sendpacketsfrom192.168.0.31to192.168.0.10: ping192.168.0.10(from192.168.0.31) Wewillsee(on192.168.0.31): From192.168.0.121:icmp_seq=2RedirectHost(New nexthop:192.168.0.10) now,runningon192.168.0.121: routeCn|grep.10 showsthatthereisanewentryintheroutingcache: 62. ICMPredirectmessagecontd. 192.168.0.31192.168.0.10192.168.0.10ri0034eth0 Therintheflagscolumnmeans:RTCF_DOREDIRECT. The192.168.0.121machinehadsentaredirectbycalling ip_rt_send_redirect()fromip_forward().(net/ipv4/ip_forward.c) 63. ICMPredirectmessagecontd. Andon192.168.0.31,runningrouteC|grep.10shows nowanewentryintheroutingcache:(incase accept_redirects=1) 192.168.0.31192.168.0.10192.168.0.10001 eth0 Incaseaccept_redirects=0(on192.168.0.31),wewillsee: 192.168.0.31192.168.0.10192.168.0.121000eth0 whichmeansthatthegwisstill192.168.0.121(whichisthe 64. ICMPredirectmessagecontd. Addinganentrytotheroutingcacheasaresultofgetting ICMPREDIRECTisdoneinip_rt_redirect(),net/ipv4/route.c. Theentryintheroutingtableisnotdeleted. 65. NeighboringSubsystem Mostknownprotocol:ARP(inIPV6:ND,neighbourdiscovery) ARPtable. Ethernetheaderis14byteslong: Sourcemacaddress(6bytes). Destinationmacaddress(6bytes). Type(2bytes). 0x0800isthetypeforIPpacket(ETH_P_IP) 0x0806isthetypeforARPpacket(ETH_P_ARP) see:include/linux/if_ether.h 66. NeighboringSubsystemcontd WhenthereisnoentryintheARPcacheforthedestinationIP addressofapacket,abroadcastissent(ARPrequest, ARPOP_REQUEST:whohasIPaddressx.y.z...).Thisisdoneby amethodcalledarp_solicit().(net/ipv4/arp.c) Youcanseethecontentsofthearptablebyrunning:cat/proc/net/arporbyrunningthearpfromacommandline. Youcandeleteandaddentriestothearptable;seemanarp. 67. BridgingSubsystem Youcandefineabridgeandadd NICstoit(enslaving ports)usingbrctl(frombridgeutils). Youcanhaveupto1024portsforeverybridgedevice (BR_MAX_PORTS). Example: brctladdbrmybr brctladdifmybreth0 brctlshow 68. BridgingSubsystemcontd. WhenaNICisconfiguredasabridgeport,thebr_port memberofnet_deviceisinitialized. (br_portisaninstanceofstructnet_bridge_port). Whenwereceiveaframe,netif_receive_skb()calls handle_bridge(). 69. BridgingSubsystemcontd. ThebridgingforwardingdatabaseissearchedforthedestinationMACaddress. Incaseofahit,theframeissenttothebridgeportwith br_forward()(net/bridge/br_forward.c). Ifthereisamiss,theframeisfloodedonallbridgeportsusingbr_flood()(net/bridge/br_forward.c). Note:thisisnotabroadcast! TheebtablesmechanismistheL2parallelofL3Netfilter. 70. BridgingSubsystemcontd Ebtablesenableustofilterandmanglepacketsatthelinklayer(L2). 71. IPSec WorksatnetworkIPlayer(L3) UsedinmanyformsofsecurednetworkslikeVPNs. MandatoryinIPv6.(notinIPv4) Implementedinmanyoperatingsystems:Linux,Solaris,Windows, andmore. RFC2401 In2.6kernel:implementedbyDaveMillerandAlexeyKuznetsov. Transformationbundles. Chainofdstentries;onlythelastoneisforrouting. 72. IPSeccont. Userspacetools:http://ipsectools.sf.net BuildingVPN:http://www.openswan.org/(OpenSource). TherearealsononIPSecsolutionsforVPN example:pptp structxfrm_policyhasthefollowingmember: structdst_entry*bundles. __xfrm4_bundle_create()createsdst_entries(withtheDST_NOHASHflag)see:net/ipv4/xfrm4_policy.c TransportModeandTunnelMode. 73. IPSeccontd. Showthesecuritypolicies: ipxfrmpolicyshow CreateRSAkeys: ipsecrsasigkeyverbose2048>keys.txt ipsecshowhostkeyleft>left.publickey ipsecshowhostkeyright>right.publickey 74. IPSeccontd. Example:HosttoHostVPN(usingopenswan) in/etc/ipsec.conf:connlinuxtolinux left=192.168.0.189 leftnexthop=%direct leftrsasigkey=0sAQPPQ... right=192.168.0.45 rightnexthop=%direct rightrsasigkey=0sAQNwb... type=tunnel auto=start 75. IPSeccontd. serviceipsecstart(tostarttheservice) ipsecverifyCheckyoursystemtoseeifIPsecgotinstalledand startedcorrectly. ipsecautostatus IfyouseeIPsecSAestablished,thisimpliessuccess. Lookforerrorsin/var/log/secure(fedoracore)orinkernelsyslog 76. Tipsforhacking Documentation/networking/ipsysctl.txt:networkingkerneltunabels Exampleofreadingahexaddress: iph>daddr==0x0A00A8C0ormeanscheckingiftheaddressis192.168.0.10(C0=192,A8=168, 00=0,0A=10). 77. TipsforhackingContd. Disablepingreply: echo1>/proc/sys/net/ipv4/icmp_echo_ignore_all Disablearp:iplinkseteth0arpoff(theNOARPflagwillbeset) Alsoifconfigeth0arphasthesameeffect. HowcanyougetthePathMTUtoadestination(PMTU)? Usetracepath(seemantracepath). Tracepathisfromiputils. 78. TipsforhackingContd. Keepiphdrstructhandy(printout):(fromlinux/ip.h) structiphdr{__u8 ihl:4, version:4; __u8 tos; __be16 tot_len; __be16 id; __be16 frag_off; __u8 ttl; __u8 protocol; __sum16check; __be32 saddr; __be32 daddr; /*Theoptionsstarthere.*/}; 79. TipsforhackingContd. NIPQUAD():macroforprintinghexaddresses CONFIG_NET_DMAisforTCP/IPoffload. Whenyouencounter:xfrm/CONFIG_XFRMthishastotodowith IPSEC.(transformers). 80. Newandfuturetrends IO/AT. NetChannels(VanJacobsonandEvgeniyPolyakov). TCPOffloading. RDMA. Mulitqueus.:somenewnics,likee1000andIPW2200,allowtwoormorehardwareTxqueues.Therearealreadypatchestoenablethis. 81. Newandfuturetrendscontd. See:EnablingLinuxNetworkSupportofHardware MultiqueueDevices,OLS2007. Somemoreinfoin:Documentation/networking/multiqueue.txt inrecentLinuxkernels. DeviceswithmultipleTX/RXqueueswillhavethe NETIF_F_MULTI_QUEUEfeature(include/linux/netdevice.h) MQnicdriverswillcallalloc_etherdev_mq()or alloc_netdev_mq()insteadofalloc_etherdev()or alloc_netdev(). 82. Linksandmoreinfo1)LinuxNetworkStackWalkthrough(2.4.20):http://gicl.cs.drexel.edu/people/sevy/network/Linux_network_stack_walkth2)UnderstandingtheLinuxKernel,SecondEditionByDanielP.Bovet,MarcoCesatiSecondEditionDecember2002chapter18:networking.UnderstandingLinuxNetworkInternals,Christianbenvenuti Oreilly,FirstEdition. 83. Linksandmoreinfo3)LinuxDeviceDriver,byJonathanCorbet,AlessandroRubini,GregKroahHartmanThirdEditionFebruary2005. Chapter17,NetworkDrivers4)Linuxnetworking:(alotofdocsaboutspecificnetworkingtopics) http://linuxnet.osdl.org/index.php/Main_Page 5)netdevmailinglist:http://www.spinics.net/lists/netdev/ 84. Linksandmoreinfo6)Removalofmultipathroutingcachefromkernelcode:http://lists.openwall.net/netdev/2007/03/12/76 http://lwn.net/Articles/241465/7)LinuxAdvancedRouting&TrafficControl:http://lartc.org/8)ebtablesafilteringtoolforabridging:http://ebtables.sourceforge.net/ 85. Linksandmoreinfo9)WritingNetworkDeviceDriverforLinux:(article) http://app.linux.org.mt/article/writingnetdrivers?locale=en 86. Linksandmoreinfo10)Netconfayearlynetworkingconference;firstwasin2004. http://vger.kernel.org/netconf2004.html http://vger.kernel.org/netconf2005.html http://vger.kernel.org/netconf2006.html Nextone:LinuxConfAustralia,January2008,Melbourne DavidS.Miller,JamesMorris,RustyRussell,JamalHadiSalim,StephenHemminger,HaraldWelte,HideakiYOSHIFUJI,HerbertXu,ThomasGraf,RobertOlsson,ArnaldoCarvalhodeMeloandothers 87. Linksandmoreinfo11)PolicyRoutingWithLinuxOnlineBookEdition byMatthewG.Marsh(Sams). http://www.policyrouting.org/PolicyRoutingBook/ 12)THRASHAdynamicLCtrieandhashdatastructure:RobertOlssonStefanNilsson,August2006http://www.csc.kth.se/~snilsson/public/papers/trash/trash.pdf13)IPSechowto:http://www.ipsechowto.org/t1.html 88. Linksandmoreinfo14)Openswan:BuildingandIntegratingVirtualPrivate Networks,byPaulWouters,KenBantofthttp://www.packtpub.com/book/openswan/mid/061205jqdnh2bypublisher:PacktPublishing.