Analysis of Botnet Security Threats

Analysis of Botnet Security ThreatsCHAPTER 1INTRODUCTION1.1 IntroductionDuring the last few decades, we catch seen the dramatically rise of the profit and its applications to the header which they catch locomote a captious kick downstairs of our lives. Internet security in that way has become more than and more primary(prenominal) to those who utilise the Internet for work, business, recreation or education.Most of the attacks and poisonous activities on the Internet be carried out by malicious applications such as Malw ar, which embroils vir procedures, trojan, worms, and botnets. Botnets become a important(prenominal) antecedent of closely of the malicious activities such as s stoogening, distrisolelyed defensive measure-of- service of process (DDoS) activities, and malicious activities overhaul across the Internet.1.2 Botnet Largest Security ThreatA bot is a softw atomic matter 18 formula, or a malw ar that runs automatically on a compromised machine without the users permission. The bot code is unremarkably written by m all criminal groups. The term bot refers to the compromised data processors in the electronic interlock. A botnet is inbredly a lucre of bots that ar under the control of an aggressor (BotMaster). Figure 1.1 illustrates a classifiable structure of a botnet.A bot commonly run through receipts of advance(a) malware techniques. As an recitation, a bot use roughly(prenominal) techniques like keylogger to record user individual(a) in figure of speechation like password and cutis its existence in the arranging. More importantly, a bot elicit distri entirelye itself on the internet to increase its scale to form a bot army. Recently, attackers use compromised Web hordes to contaminate those who visit the websites finished drive-by transfer 6. Currently, a botnet contains thousands of bots, exactly t here(predicate) is some cases that botnet contain several millions of bots 7. genuinely bots variediat e themselves from otherwise kind of worms by their ability to ask oer commands from attacker impertinently 32. assaulter or better call it botherder control bots by means of different protocols and structures. The Internet pass Chat (IRC) protocol is the earliest and still the close commonly utilize CC stemma at present. HTTP is excessively utilize be give Http protocol is permitted in nearly nets. centralize structure botnets was rattling triple-crown in the past but promptly botherders use change structure to avoid single point of failure riddle.Unlike previous malware such as worms, which are use belike for entertaining, botnets are utilise for real financial abuse. Actually Botnets bay window cause m whatever problems as some of them makeed infrai. Click fraud. A botmaster put up tardily simoleons by forcing the bots to click on advertisement for the purpose of personal or commercial abuse.ii. Spam production. Majority of the email on the internet is e-mail.iii. DDoS attacks. A bot army tail assembly be commanded to begin a distributed denial-of-service attack against whatever machine.iv. Phishing. Botnets are widely used to entertain malicious phishing sites. Criminals usually send junk e-mail kernels to shit users to visit their forged web sites, so that they brook obtain users critical information such as usernames, passwords.1.3 Botnet in-Depthpresent, the most(prenominal) serious manifestation of mod malware is Botnet. To father distinction amid Botnet and other kinds of malware, the concepts of Botnet swallow to understand. For a better collar of Botnet, two important terms, Bot and BotMaster accept been defined from another point of views.Bot Bot is rattling short circuit for robot which is likewise called as Zombie. It is a spick-and-span guinea pig of malware 24 installed into a compromised computer which hatful be controlled remotely by BotMaster for executing some secernates through the received commands. After the Bot code has been installed into the compromised computers, the computer becomes a Bot or Zombie 25. Contrary to existing malware such as computer computer virus and worm which their main activities emphasis on attacking the sullying phalanx, bots can receive commands from BotMaster and are used in distributed attack platform.BotMaster BotMaster is also known as BotHerder, is a person or a group of person which control remote Bots. Botnets- Botnets are networks consisting of large number of Bots. Botnets are created by the BotMaster to setup a private communication understructure which can be used for malicious activities such as Distributed Denial-of-Service (DDoS), sending large amount of SPAM or phishing mails, and other nefarious purpose 26, 27, 28. Bots spoil a persons computer in umteen ways.Bots usually disseminate themselves across the Internet by sayinging for vulnerable and unprotected computers to infect. When they retrieve an unprotected com puter, they infect it and and past send a traverse to the BotMaster. The Bot stay hidden until they are announced by their BotMaster to perform an attack or task. Other ways in which attackers use to infect a computer in the Internet with Bot include sending email and utilize malicious websites, but common way is intrusive the Internet to smell for vulnerable and unprotected computers 29. The activities associated with Botnet can be classified into three parts (1) Searching searching for vulnerable and unprotected computers. (2) Dissemination the Bot code is distributed to the computers ( signals), so the targets become Bots. (3) sign-on the Bots consort to BotMaster and become ready to receive command and control work.The main difference between Botnet and other kind of malwares is the existence of Command-and-Control (CC) infrastructure. The CC allows Bots to receive commands and malicious capabilities, as prone by BotMaster. BotMaster must(prenominal) ensure that the ir CC infrastructure is sufficiently robust to manage thousands of distributed Bots across the globe, as considerably as resisting whatsoever adjudicates to shutdown the Botnets. However, maculation and mitigation techniques against Botnets have been increased 30,31. Recently, attackers are also continually improving their approaches to protect their Botnets. The premier(prenominal) generation of Botnets utilise the IRC (Internet put across Chat) channels as their Common-and-Control (CC) centers. The modify CC mechanism of such Botnet has made them vulnerable to being followed and disabled. so, invigorated generation of Botnet which can hide their CC communication have emerged, Peer-to-Peer (P2P) solution Botnets. The P2P Botnets do not experience from a single point of failure, because they do not have centralized CC servers 35. Attackers have accordingly developed a range of strategies and techniques to protect their CC infrastructure.Therefore, considering the CC fun ction gives better understanding of Botnet and help defenders to soma proper signal encounterion or mitigation techniques. According to the CC channel we categorize Botnets into three different topologies a) Centralized b) Decentralized and c) crossbred. In Section 1.1.4, these topologies have been analyzed and totally considered the protocols that are currently being used in to all(prenominal) hotshot warning.1.4 Botnet TopologiesAccording to the Command-and-Control(CC) channel, Botnet topology is categorise into three different examples, the Centralized model, the Decentralized model and Hybrid model.1.4.1 Centralized ModelThe oldest type of topology is the centralized model. In this model, maven central point is responsible for exchanging commands and data between the BotMaster and Bots. In this model, BotMaster chooses a host (usually uplifted bandwidth computer) to be the central point (Command-and-Control) server of all the Bots. The CC server runs certain network services such as IRC or HTTP. The main utility of this model is small message latency which cause BotMaster easily arranges Botnet and launch attacks.Since all connections happen through the CC server, therefore, the CC is a critical point in this model. In other words, CC server is the weak point in this model. If somebody manages to discover and eliminates the CC server, the unblemished Botnet will be grievous-for-nothing and ineffective. Thus, it becomes the main drawback of this model. A lot of modern centralized Botnets employed a list of IP addresses of alternative CC servers, which will be used in case a CC server discovered and has been taken offline.Since IRC and HTTP are two common protocols that CC server uses for communication, we consider Botnets in this model establish on IRC and HTTP. Figure 1.2 shows the base communication architecture for a Centralized model. There are two central points that forward commands and data between the BotMaster and his Bots.1.4.1. 1 Botnets based on IRCThe IRC is a type of real-time Internet text messaging or synchronous conferencing 36. IRC protocol is based on the Client host model that can be used on many computers in distributed networks. Some advantages which made IRC protocol widely being used in remote communication for Botnets are (i) low latency communication (ii) anonymous real-time communication (iii) ability of Group (many-to-many) and Private ( matched) communication (iv) simplistic to setup and (v) simple commands. The basic commands are connect to servers, join channels and post messages in the channels (vi) very flexibility in communication. Therefore IRC protocol is still the most everyday protocol being used in Botnet communication.In this model, BotMasters can command all of their Bots or command a few of the Bots using one-to-one communication. The CC server runs IRC service that is the same with other standard IRC service. Most of the time BotMaster creates a channel on the IRC server that all the bots can connect, which inculcate apiece connected bot to do the BotMasters commands. Figure 1.3 showed that there is one central IRC server that forwards commands and data between the BotMaster and his Bots.Puri 38 presented the procedures and mechanism of Botnet based on IRC, as shown in Figure. 1.4.Bots transmission and control process 38i. The attacker tries to infect the targets with Bots.ii. After the Bot is installed on target machine, it will try to connect to IRC server. In this while a random nickname will be generate that show the bot in attackers private channel.iii. Request to the DNS server, dynamic mapping IRC servers IP address.iv. The Bot will join the private IRC channel set up by the attacker and wait for steerings from the attacker. Most of these private IRC channel is set as the encrypted mode.v. Attacker sends attack instruction in private IRC channel.vi. The attacker tries to connect to private IRC channel and send the au sotication password.v ii. Bots receive instructions and launch attacks such as DDoS attacks.1.4.1.2 Botnet based on HTTPThe HTTP protocol is an additional well-known protocol used by Botnets. Because IRC protocol within Botnets became well-known, internet security researchers gave more consideration to monitoring IRC traffic to detect Botnet. Consequently, attackers started to use HTTP protocol as a Command-and-Control communication channel to make Botnets become more difficult to detect. The main advantage of using the HTTP protocol is concealing Botnets traffics in normal web traffics, so it can easily passes firewalls and avoid IDS spying. Usually firewalls block incoming and outgoing traffic to not needed ports, which usually include the IRC port.1.4.2 Decentralized modelDue to major disadvantage of Centralized model-Central Command-and-Control (CC)-attackers tried to build another Botnet communication topology that is harder to discover and to destroy. Hence, they decided to notice a model in whi ch the communication system does not heavily depending on few selected servers and even discovering and destroying a number of Bots.As a result, attackers take advantage of Peer-to-Peer (P2P) communication as a Command-and-Control (CC) embodiment which is much harder to shut down in the network. The P2P based CC model will be used easily in Botnets in the future, and definitely Botnets that use P2P based CC model impose much bigger challenge for defense of networks.In the P2P model, as shown in Fig. 1.6, there is no Centralized point for communication. Each Bot have some connections to the other Bots of the same Botnet and Bots act as both Clients and servers. A new Bot must know some addresses of the Botnet to connect there. If Bots in the Botnet are taken offline, the Botnet can still observe to operate under the control of BotMaster.P2P Botnets aim at removing or hiding the central point of failure which is the main weakness and vulnerability of Centralized model. Some P2P Bot nets operate to a certain extent deconcentrate and some completely decentralized. Those Botnets that are completely decentralized allow a BotMaster to insert a command into any Bots. Since P2P Botnets usually allow commands to be injected at any thickener in the network, the authentication of commands become essential to prevent other bosss from injecting incorrect commands.For a better understanding in this model, some characteristics and important features of historied P2P Botnets have been mentioned Slapper Allows the routing of commands to distinct nodes. Uses Public key and private key cryptography to authenticate commands. BotMasters sign commands with private key and only those nodes which has interchangeable normal key can verify the commands 42. Two important weak points are (a) its list of known Bots contains all (or almost all) of the Botnet. Thus, one single captured Bot would give away the entire Botnet to defenders 42 (b) its sophisticated communication mechanism produces lot traffic, making it vulnerable to monitoring via network flow analysis. Sinit This Bot uses random searching to discove other Bots to communicate with. It can results in an easy detection due to the extensive probing traffic 34. Nugache Its weakness is based on its reliance on a seed list of 22 IP addresses during its bootstrap process 47. Phatbot Uses Gnutella cache server for its bootstrap process which can be easily shutdown. Also its WASTE P2P protocol has a scalability problem across a long network 48. Strom worm it uses a P2p overnet protocl to control compromised hosts. The communication protocol for this Bot can be classified into five steps, as describes below 37 i. Connect to Overnet Bots try to join Overnet network. Each Bot initially has hard-coded binary agitates which is included the IP addresses of P2P-based Botnet nodes. ii. Search and Download Secondary Injection URL Bot uses hard-coded keys to explore for and transfer the URL on the Overnet netwo rk 37. iii. Decrypt Secondary Injection URL compromised hosts take advantages of a key(hard coded) to decrypt the URL. iv. Download Secondary Injection compromised hosts attempt to download the second injection from a server(probably web server). It could be give files or updated files or list of the P2P nodes 37.1.4.3 Hybrid modelThe Bots in the Hybrid Botnet are categorized into two groups1) Servant Bots Bots in the first group are called as servant Bots, because they behave as both clients and servers, which have static, routable IP addresses and are neighborly from the entire Internet.2) Client Bots Bots in the second group is called as client Bots since they do not accept incoming connections. This group contains the remaining Bots, including- (a) Bots with dynamically designated IP addresses (b) Bots with Non-routable IP addresses and (c) Bots behind firewalls which they cannot be connected from the global Internet.1.5 Background of the ProblemBotnets which are controlle d remotely by BotMasters can launch huge denial of service attacks, several infiltration attacks, can be used to spread spam and also conduct malicious activities 115. While bot army activity has, so far, been exceptional to criminal activity, their potential for causing large- scale damage to the entire internet is immeasurable 115. Therefore, Botnets are one of the most dangerous types of network-based attack today because they posit the use of very large, synchronized groups of hosts for their malicious activities.Botnets obtain their power by size, both in their increasing bandwidth and in their reach. As mentioned before Botnets can cause severe network disruptions through huge denial- of-service attacks, and the danger of this interruption can charge enterprises big sums in extortion fees. Botnets are also used to harvest-feast personal, corporate, or government sensitive information for sale on a blooming organized crime market.1.6 Statement of the ProblemRecently, botnet s are using new type of command-and-control(CC) communication which is totally decentralized. They utilize peer-to-peer style communication. Tracking the starting point and activity of this botnet is much more complicated due to the Peer-to-Peer communication infrastructure.Combating botnets is usually an turn up of discovering their weakness their central position of command, or CC server. This is typically an IRC network that all bots connect to central point, however with the use of P2P mode we cannot find any central point of command. In the P2P networks each bots in searching to connect other peers which can receive or broadcast commands through network. Therefore, an dead-on(prenominal) detection and fighting method is required to prevent or stop such dangerous networks.1.7 Research Questionsa. What are the main differences between centralized and decentralized botnets?b. What is the vanquish and efficient general extensible solution for detecting non-specific Peer-to- Pe er botnets?1.8 Objectives of the Studyi. To develop a network-based framework for Peer-to-Peer botnets detection by common appearance in network communication.ii. To study the conduct of bots and recognizing portal similarities across multiple bots in order to develop mentioned framework.1.9 Scope of the StudyThe project scope is hold to ontogeny some algorithms pertaining to our proposed framework. This algorithms are using for decreasing traffics by filtering it, classifying intended traffics, monitoring traffics and the detection of malicious activities.1.10 Significance of the studyPeer-to-Peer botnets are one of the most sophisticated types of cyber crime today. They give the full control of many computers around to world to exploit them for malicious activities purpose such as spread of virus and worm, spam distribution and DDoS attack. Therefore, studying the behavior of P2P botnets and develop a technique that can detect them is important and high-demanded.1.11 SummaryUn derstanding the Botnet Command-and-Control(CC) is a critical part in recognizing how to best protect against the overall botnet threat. The CC channels utilized by the Botnets will often show the type and degree of actions an enterprise can see in either blocking or shutting down a botnet, and the probability of success.It is also obvious that attackers have been trying for years to strike away from Centralized CC channels, and are achieving some success using Decentralized(P2P) CC channels over the last 5 or so years. Therefore in this chapter we have defined a classification for better understanding of Botnets CC channels, which is included Centralized, Decentralized, and Hybrid model and tried to evaluate recognized protocols in each of them. Understanding the communication topologies in Botnets is essential to precisely identify, detect and mitigate the ever-increasing Botnets threats.CHAPTER 2LITERATURE REVIEW2.1 IntroductionBefore majority of botnets was using IRC (Internet Relay Chat) as a communication protocol for Command and Control(CC) mechanism. Therefore, many researches tried to develop botnet detection scheme which was based on analysis of IRC traffic 50. As a result, attackers decided to develop more sophisticated botnets, such as Storm worm and Nugache toward the utilization of P2P networks for CC infrastructures. In response to this movement, researches have proposed conglomerate models of botnets detection that are based on P2P infrastructure 5.One key advantage of both IRC and HTTP Botnet is the use of central Command and Control. This characteristic provides the attacker with very well-organized communication. However, the assets also considers as a main disadvantage to the attacker 8. The threat of the Botnet can be decreased and possibly omitted if the central CC is taken over or taken down 8. The method that is starting to come out is P2P structure for Botnet interaction. There is not any centralized centre for P2P botnets. whatsoe ver nodes in P2P botnet behave as client and server as well. If any point in the network is shut down the botnet still can continue its operation.The wedge botnet is one of the main and recognized recent P2P botnets. It customized the overnet P2P file- overlap application which is based on the Kademlia distributed hash table algorithm 55 and exploit it for its CC infrastructure. Recently many researchers specially in the anti-virus community and electronic media concentrated on storm worm 56,57.2.2 Background and HistoryA peer-to-peer network is a network of computers that any computer in the network can behave as both a client and a server.Some explanation of peer-to-peer networks does not need any form of centralized coordination. This definition is more comfortable because the attacker may be interested in hybrid architectures 8.2.2.1 HistoryThe table 2.1 shows a compendium of some well-known bots and P2P protocols. The range of time from the first bots, EggDrop, until the Sto rm Worm P2P bot is newly released. The first non-malicious bot was EggDrop that came up many years ago, and we know it as one of the first IRC bots that came to market. GTBot that have many other categories is another well-known malicious bot, that its variants are IRC client, mIRC.exe61.After a while, P2P protocols have been used for Botnet activities. Napster is one of the first bot that used P2P as its communication. Napster built an platform that permit all bots can find each other and share files with each other in the network. In this bot, file sharing has been done in the centralized server that we can say it was not completely a P2P botnet. Therefore, all bots have to upload an index of their files to the centralized server and also if they are aspect for other files among all bots, have to search in centralized server. If it can find any file that looking for, then can directly connect to that bot and download what they compliments. Nowadays, because Napster has been shut down as their service recognized as illegal service, many other P2P service counseling on avoiding such finding.After few years after Napster, Gnutella protocol came up as the first completely P2P services. Actually after Gnutellas , as shown in hedge 2.1, many other P2P protocols have been released, such as Kademilia and Chord. This two new p2p service are using distributed hash table as a method for finding information in the peer-to-peer networks.Agobot is another malicious P2P bot that came up recently and become widespread because of dandy design and modular code base 61. Nowadays many researchers are concentrating on P2P bots and there is an anticipation that P2P bots will reach to the stage that Centralized botnets will not been used any more in the future.Table 2.1 P2P based Botnets2.3 Peers-to-Peer Overlay NetworksOverlay networks are categorized into two categories Structured and Unstructured. All nodes in first category can connect to most X peers regarding some condit ions for identification of nodes that those peers want to connect. However in unstructured type there is not any specified limit for the number of peers that they can connect, in spite of the fact that there is not any condition for connecting to other peers. Overnet is a good example of structured p2p networks and Chorf is a good example of unstructured P2P networks.2.3.1 Brief overview of OvernetOne of the popular file sharing networks is Overnet that use for their design use distributed hash table (DHT) algorithm that called Kademlia55. Each node produces a 128-bit id for joining the network and also use for sending to other node for introducing itself. Actually each node in the network saves the information astir(predicate) other nodes in order to route query messages.2.3.2 Brief overview of GnutellaGnutellas is a unstructured file sharing network. In this network, when a node like n want to connect to a node like m, use a ping message to inform the other node for its presence. As long as node m received ping message, then send it back to other nodes in its neighbor and also send a Pong message to the vector of ping message that was node n. this transaction among node let them to learn about each other.2.4 Botnet spyingIn particular, to compare existing botnet detection techniques, different methods are described and then disadvantages of each method are mentioned respectively.2.4.1 Honeypot-based trackHoneypot can be used to collect bots for analyzing its behavior and hints and also for tracking botnets. But using honeypots have several limitations. The most important limitation is because of limited scale of exploited activities that can track. And also it cannot capture the bots that use the method of propagation other than interpretning, such as spam. And finally it can only give report for transmission machines that are anticipated and put in the network as trap system. So it means that it can not give a report for those computers that are giv e with bot in the network but are not devoted as trap machines. So we can come to this conclusion that generally in this technique we have to wait until one bot in the network infect our system and then we can track or analyze the machine.2.4.2 Intrusion detection systemsIntrusion detection techniques can be categorized into two categories host-based and network-based solution. Host-based techniques are used for recognizing malware binaries such as viruses. A good example of this type is anti-virus detection systems. However, we know that anti-virus are good for fair(a) virus detection. The most important disadvantages of anti-virus are that bots can easily evade the detection technique by changing their signatures easily, because the detection system cannot update their databases consistency. And also bots can disable any anti-virus tools in the system to protect themselves from detection.Network- based intrusion detection system is another method for detection that is used in the area of botnet detection. Snort67 and Bro68 are the two well-known signature based detection system that are used currently. They use a database as signatures of famous malicious activities to detect botnets or any other malware. Actually if our objective is using this technique for botnet detection, we have to keep updating the database and recognizing all malware quickly to make a signature of it and add to our database. For solving this solving this problem recently researchers are using unusual person based IDS that can detect malicious activities based on behavior of malware or detection techniques.2.4.3 Bothunter Dialog correlation-based Botnet detectionThis technique developed an evidence-trail approach for detecting successful bot infection with patterns during communication for infection process. In this strategy, bot infection pattern are modeled to use for recognizing the whole process of infection of botnet in the network. All behavior that occur the bot infection su ch as target scanning, CC establishment, binary downloading and outbound propagation have to model by this method. This method gathers an evidence-trail of connected infection process for each internal machine and then tries to look for a verge combination of sequences that will convince the condition for bot infection 32.The BotHunter use snort with adding two anomaly-detection components to it that are SLADE (Statistical payLoad Anomaly Detection Engine) and SCADE (Statistical scan Anomaly Detection Engine). SCADE produce internal and external scan detection warnings that are weighted for criticality toward malware scanning patterns. SLADE perform a byte-distribution onus anomaly detection of incoming packets, providing a matching non-signature approach in inbound exploit detection 32 .Slade use an n-gram payload examination of traffics that have typical malware intrusions. SCADE persist some port scan analysis for incoming and outgoing traffics. Actually BotHunter has a link be tween scan and alarm intrusion that shows a host has been infected. When a adequate sequence of alerts is established to match BotHunters infection dialog model, a comprehensive report is created to get all the related events participants that have a rule in infection dialog 32. This method provides some important featuresi. This technique concentrates on malware detection by IDS-driven dialog correlation. This model shows an essential network processes that occur during a successful bot infection.ii. This technique has one IDS-independent dialog correlation engine and three bot-specific sensors. This technique can automatically produce a report of whole detection of bot, as well as the infection of agent, identification of the computer that has been infected and source of Command and Control centre.2.4.3.1 Bot infection sequencesActually understanding bot infection life processes is a challenging work for protection of network in the future. The major work in this area is different iating between successful bot infection and background exploit attempt. For reaching to this point analysis of two-way dialog flow between internal hosts and external hosts (internet) is needed. In a good design network which uses filtering at gateway, the threats of direct exploitations are limited. However, contemporary malware families are highly flexible in their ability to attack vulnerable hosts through email attachments, infected P2P media, and drive-by download infections 32.2.4.3.2 Modeling the infection dialog processThe bot distribution model can conclude by an analysis of external communication traffics that shows the behavior of pertinent botnet. Incoming scan and utilize alarms are not enough to state a winning malware infection, as are assumed that a lasting sprout of scan and exploit signals will be observed from the way out monitor 32.Figure 2.1 shows the process of bot infection in BotHunter that used for evaluating network flows through eight stages. This model is almost similar with the model that Rajab et al. presented for IRC detection model. The model that they proposed has early initial scanning that is a preceding consideration happen in form of IP exchange and pointing vulnerable ports. Actually figure 2.1 is not aimed for a strict ordering of infection events that happen during bot infection.The important resultant here is that bot dialog processes analysis have to be strong to the absence of some dialog events and must not need strong sequencing on the order in bound dialog is conducted. One solution to solve the problem of sequence order and event is to use a weighted event threshold system that take smallest essential sparse sequences of events under which bot profile statement can be initiated 32. For instance, it is possible put weighting and threshold system for the look of each event in a way that a smallest set of event is important prior of bot detection.2.4.3.3 Design and implementationMore attention devoted for shrewd a passive network monitoring system in this part which be able of identifying the bidirectional warning signs when internal hosts are infected with bAnalysis of Botnet Security ThreatsAnalysis of Botnet Security ThreatsCHAPTER 1INTRODUCTION1.1 IntroductionDuring the last few decades, we have seen the dramatically rise of the Internet and its applications to the point which they have become a critical part of our lives. Internet security in that way has become more and more important to those who use the Internet for work, business, entertainment or education.Most of the attacks and malicious activities on the Internet are carried out by malicious applications such as Malware, which includes viruses, trojan, worms, and botnets. Botnets become a main source of most of the malicious activities such as scanning, distributed denial-of-service (DDoS) activities, and malicious activities happen across the Internet.1.2 Botnet Largest Security ThreatA bot is a software code, or a malware th at runs automatically on a compromised machine without the users permission. The bot code is usually written by some criminal groups. The term bot refers to the compromised computers in the network. A botnet is essentially a network of bots that are under the control of an attacker (BotMaster). Figure 1.1 illustrates a typical structure of a botnet.A bot usually take advantage of sophisticated malware techniques. As an example, a bot use some techniques like keylogger to record user private information like password and hide its existence in the system. More importantly, a bot can distribute itself on the internet to increase its scale to form a bot army. Recently, attackers use compromised Web servers to contaminate those who visit the websites through drive-by download 6. Currently, a botnet contains thousands of bots, but there is some cases that botnet contain several millions of bots 7.Actually bots differentiate themselves from other kind of worms by their ability to receive c ommands from attacker remotely 32. Attacker or better call it botherder control bots through different protocols and structures. The Internet Relay Chat (IRC) protocol is the earliest and still the most commonly used CC channel at present. HTTP is also used because Http protocol is permitted in most networks. Centralized structure botnets was very successful in the past but now botherders use decentralized structure to avoid single point of failure problem.Unlike previous malware such as worms, which are used probably for entertaining, botnets are used for real financial abuse. Actually Botnets can cause many problems as some of them listed belowi. Click fraud. A botmaster can easily profit by forcing the bots to click on advertisement for the purpose of personal or commercial abuse.ii. Spam production. Majority of the email on the internet is spam.iii. DDoS attacks. A bot army can be commanded to begin a distributed denial-of-service attack against any machine.iv. Phishing. Botnets are widely used to host malicious phishing sites. Criminals usually send spam messages to deceive users to visit their forged web sites, so that they can obtain users critical information such as usernames, passwords.1.3 Botnet in-DepthNowadays, the most serious manifestation of advanced malware is Botnet. To make distinction between Botnet and other kinds of malware, the concepts of Botnet have to understand. For a better understanding of Botnet, two important terms, Bot and BotMaster have been defined from another point of views.Bot Bot is actually short for robot which is also called as Zombie. It is a new type of malware 24 installed into a compromised computer which can be controlled remotely by BotMaster for executing some orders through the received commands. After the Bot code has been installed into the compromised computers, the computer becomes a Bot or Zombie 25. Contrary to existing malware such as virus and worm which their main activities focus on attacking the infe cting host, bots can receive commands from BotMaster and are used in distributed attack platform.BotMaster BotMaster is also known as BotHerder, is a person or a group of person which control remote Bots. Botnets- Botnets are networks consisting of large number of Bots. Botnets are created by the BotMaster to setup a private communication infrastructure which can be used for malicious activities such as Distributed Denial-of-Service (DDoS), sending large amount of SPAM or phishing mails, and other nefarious purpose 26, 27, 28. Bots infect a persons computer in many ways.Bots usually disseminate themselves across the Internet by looking for vulnerable and unprotected computers to infect. When they find an unprotected computer, they infect it and then send a report to the BotMaster. The Bot stay hidden until they are announced by their BotMaster to perform an attack or task. Other ways in which attackers use to infect a computer in the Internet with Bot include sending email and usin g malicious websites, but common way is searching the Internet to look for vulnerable and unprotected computers 29. The activities associated with Botnet can be classified into three parts (1) Searching searching for vulnerable and unprotected computers. (2) Dissemination the Bot code is distributed to the computers (targets), so the targets become Bots. (3) sign-on the Bots connect to BotMaster and become ready to receive command and control traffic.The main difference between Botnet and other kind of malwares is the existence of Command-and-Control (CC) infrastructure. The CC allows Bots to receive commands and malicious capabilities, as devoted by BotMaster. BotMaster must ensure that their CC infrastructure is sufficiently robust to manage thousands of distributed Bots across the globe, as well as resisting any attempts to shutdown the Botnets. However, detection and mitigation techniques against Botnets have been increased 30,31. Recently, attackers are also continually impr oving their approaches to protect their Botnets. The first generation of Botnets utilized the IRC (Internet Relay Chat) channels as their Common-and-Control (CC) centers. The centralized CC mechanism of such Botnet has made them vulnerable to being detected and disabled. Therefore, new generation of Botnet which can hide their CC communication have emerged, Peer-to-Peer (P2P) based Botnets. The P2P Botnets do not experience from a single point of failure, because they do not have centralized CC servers 35. Attackers have accordingly developed a range of strategies and techniques to protect their CC infrastructure.Therefore, considering the CC function gives better understanding of Botnet and help defenders to design proper detection or mitigation techniques. According to the CC channel we categorize Botnets into three different topologies a) Centralized b) Decentralized and c) Hybrid. In Section 1.1.4, these topologies have been analyzed and completely considered the protocols that are currently being used in each model.1.4 Botnet TopologiesAccording to the Command-and-Control(CC) channel, Botnet topology is categorized into three different models, the Centralized model, the Decentralized model and Hybrid model.1.4.1 Centralized ModelThe oldest type of topology is the centralized model. In this model, one central point is responsible for exchanging commands and data between the BotMaster and Bots. In this model, BotMaster chooses a host (usually high bandwidth computer) to be the central point (Command-and-Control) server of all the Bots. The CC server runs certain network services such as IRC or HTTP. The main advantage of this model is small message latency which cause BotMaster easily arranges Botnet and launch attacks.Since all connections happen through the CC server, therefore, the CC is a critical point in this model. In other words, CC server is the weak point in this model. If somebody manages to discover and eliminates the CC server, the entire Botne t will be worthless and ineffective. Thus, it becomes the main drawback of this model. A lot of modern centralized Botnets employed a list of IP addresses of alternative CC servers, which will be used in case a CC server discovered and has been taken offline.Since IRC and HTTP are two common protocols that CC server uses for communication, we consider Botnets in this model based on IRC and HTTP. Figure 1.2 shows the basic communication architecture for a Centralized model. There are two central points that forward commands and data between the BotMaster and his Bots.1.4.1.1 Botnets based on IRCThe IRC is a type of real-time Internet text messaging or synchronous conferencing 36. IRC protocol is based on the Client Server model that can be used on many computers in distributed networks. Some advantages which made IRC protocol widely being used in remote communication for Botnets are (i) low latency communication (ii) anonymous real-time communication (iii) ability of Group (many-to-m any) and Private (one-to-one) communication (iv) simple to setup and (v) simple commands. The basic commands are connect to servers, join channels and post messages in the channels (vi) very flexibility in communication. Therefore IRC protocol is still the most popular protocol being used in Botnet communication.In this model, BotMasters can command all of their Bots or command a few of the Bots using one-to-one communication. The CC server runs IRC service that is the same with other standard IRC service. Most of the time BotMaster creates a channel on the IRC server that all the bots can connect, which instruct each connected bot to do the BotMasters commands. Figure 1.3 showed that there is one central IRC server that forwards commands and data between the BotMaster and his Bots.Puri 38 presented the procedures and mechanism of Botnet based on IRC, as shown in Figure. 1.4.Bots infection and control process 38i. The attacker tries to infect the targets with Bots.ii. After the Bot is installed on target machine, it will try to connect to IRC server. In this while a random nickname will be generate that show the bot in attackers private channel.iii. Request to the DNS server, dynamic mapping IRC servers IP address.iv. The Bot will join the private IRC channel set up by the attacker and wait for instructions from the attacker. Most of these private IRC channel is set as the encrypted mode.v. Attacker sends attack instruction in private IRC channel.vi. The attacker tries to connect to private IRC channel and send the authentication password.vii. Bots receive instructions and launch attacks such as DDoS attacks.1.4.1.2 Botnet based on HTTPThe HTTP protocol is an additional well-known protocol used by Botnets. Because IRC protocol within Botnets became well-known, internet security researchers gave more consideration to monitoring IRC traffic to detect Botnet. Consequently, attackers started to use HTTP protocol as a Command-and-Control communication channel to ma ke Botnets become more difficult to detect. The main advantage of using the HTTP protocol is hiding Botnets traffics in normal web traffics, so it can easily passes firewalls and avoid IDS detection. Usually firewalls block incoming and outgoing traffic to not needed ports, which usually include the IRC port.1.4.2 Decentralized modelDue to major disadvantage of Centralized model-Central Command-and-Control (CC)-attackers tried to build another Botnet communication topology that is harder to discover and to destroy. Hence, they decided to find a model in which the communication system does not heavily depending on few selected servers and even discovering and destroying a number of Bots.As a result, attackers take advantage of Peer-to-Peer (P2P) communication as a Command-and-Control (CC) pattern which is much harder to shut down in the network. The P2P based CC model will be used considerably in Botnets in the future, and definitely Botnets that use P2P based CC model impose much bi gger challenge for defense of networks.In the P2P model, as shown in Fig. 1.6, there is no Centralized point for communication. Each Bot have some connections to the other Bots of the same Botnet and Bots act as both Clients and servers. A new Bot must know some addresses of the Botnet to connect there. If Bots in the Botnet are taken offline, the Botnet can still continue to operate under the control of BotMaster.P2P Botnets aim at removing or hiding the central point of failure which is the main weakness and vulnerability of Centralized model. Some P2P Botnets operate to a certain extent decentralized and some completely decentralized. Those Botnets that are completely decentralized allow a BotMaster to insert a command into any Bots. Since P2P Botnets usually allow commands to be injected at any node in the network, the authentication of commands become essential to prevent other nodes from injecting incorrect commands.For a better understanding in this model, some characteristic s and important features of famous P2P Botnets have been mentioned Slapper Allows the routing of commands to distinct nodes. Uses Public key and private key cryptography to authenticate commands. BotMasters sign commands with private key and only those nodes which has corresponding public key can verify the commands 42. Two important weak points are (a) its list of known Bots contains all (or almost all) of the Botnet. Thus, one single captured Bot would expose the entire Botnet to defenders 42 (b) its sophisticated communication mechanism produces lot traffic, making it vulnerable to monitoring via network flow analysis. Sinit This Bot uses random searching to discove other Bots to communicate with. It can results in an easy detection due to the extensive probing traffic 34. Nugache Its weakness is based on its reliance on a seed list of 22 IP addresses during its bootstrap process 47. Phatbot Uses Gnutella cache server for its bootstrap process which can be easily shutdown. Also its WASTE P2P protocol has a scalability problem across a long network 48. Strom worm it uses a P2p overnet protocl to control compromised hosts. The communication protocol for this Bot can be classified into five steps, as describes below 37 i. Connect to Overnet Bots try to join Overnet network. Each Bot initially has hard-coded binary files which is included the IP addresses of P2P-based Botnet nodes. ii. Search and Download Secondary Injection URL Bot uses hard-coded keys to explore for and download the URL on the Overnet network 37. iii. Decrypt Secondary Injection URL compromised hosts take advantages of a key(hard coded) to decrypt the URL. iv. Download Secondary Injection compromised hosts attempt to download the second injection from a server(probably web server). It could be infected files or updated files or list of the P2P nodes 37.1.4.3 Hybrid modelThe Bots in the Hybrid Botnet are categorized into two groups1) Servant Bots Bots in the first group are called as ser vant Bots, because they behave as both clients and servers, which have static, routable IP addresses and are accessible from the entire Internet.2) Client Bots Bots in the second group is called as client Bots since they do not accept incoming connections. This group contains the remaining Bots, including- (a) Bots with dynamically designated IP addresses (b) Bots with Non-routable IP addresses and (c) Bots behind firewalls which they cannot be connected from the global Internet.1.5 Background of the ProblemBotnets which are controlled remotely by BotMasters can launch huge denial of service attacks, several infiltration attacks, can be used to spread spam and also conduct malicious activities 115. While bot army activity has, so far, been limited to criminal activity, their potential for causing large- scale damage to the entire internet is immeasurable 115. Therefore, Botnets are one of the most dangerous types of network-based attack today because they involve the use of very la rge, synchronized groups of hosts for their malicious activities.Botnets obtain their power by size, both in their increasing bandwidth and in their reach. As mentioned before Botnets can cause severe network disruptions through huge denial- of-service attacks, and the danger of this interruption can charge enterprises big sums in extortion fees. Botnets are also used to harvest personal, corporate, or government sensitive information for sale on a blooming organized crime market.1.6 Statement of the ProblemRecently, botnets are using new type of command-and-control(CC) communication which is totally decentralized. They utilize peer-to-peer style communication. Tracking the starting point and activity of this botnet is much more complicated due to the Peer-to-Peer communication infrastructure.Combating botnets is usually an issue of discovering their weakness their central position of command, or CC server. This is typically an IRC network that all bots connect to central point, how ever with the use of P2P method we cannot find any central point of command. In the P2P networks each bots in searching to connect other peers which can receive or broadcast commands through network. Therefore, an accurate detection and fighting method is required to prevent or stop such dangerous networks.1.7 Research Questionsa. What are the main differences between centralized and decentralized botnets?b. What is the best and efficient general extensible solution for detecting non-specific Peer-to- Peer botnets?1.8 Objectives of the Studyi. To develop a network-based framework for Peer-to-Peer botnets detection by common behavior in network communication.ii. To study the behavior of bots and recognizing behavioral similarities across multiple bots in order to develop mentioned framework.1.9 Scope of the StudyThe project scope is limited to developing some algorithms pertaining to our proposed framework. This algorithms are using for decreasing traffics by filtering it, classifyin g intended traffics, monitoring traffics and the detection of malicious activities.1.10 Significance of the studyPeer-to-Peer botnets are one of the most sophisticated types of cyber crime today. They give the full control of many computers around to world to exploit them for malicious activities purpose such as spread of virus and worm, spam distribution and DDoS attack. Therefore, studying the behavior of P2P botnets and develop a technique that can detect them is important and high-demanded.1.11 SummaryUnderstanding the Botnet Command-and-Control(CC) is a critical part in recognizing how to best protect against the overall botnet threat. The CC channels utilized by the Botnets will often show the type and degree of actions an enterprise can follow in either blocking or shutting down a botnet, and the probability of success.It is also obvious that attackers have been trying for years to move away from Centralized CC channels, and are achieving some success using Decentralized(P2P) CC channels over the last 5 or so years. Therefore in this chapter we have defined a classification for better understanding of Botnets CC channels, which is included Centralized, Decentralized, and Hybrid model and tried to evaluate recognized protocols in each of them. Understanding the communication topologies in Botnets is essential to precisely identify, detect and mitigate the ever-increasing Botnets threats.CHAPTER 2LITERATURE REVIEW2.1 IntroductionBefore majority of botnets was using IRC (Internet Relay Chat) as a communication protocol for Command and Control(CC) mechanism. Therefore, many researches tried to develop botnet detection scheme which was based on analysis of IRC traffic 50. As a result, attackers decided to develop more sophisticated botnets, such as Storm worm and Nugache toward the utilization of P2P networks for CC infrastructures. In response to this movement, researches have proposed various models of botnets detection that are based on P2P infrastructure 5.One key advantage of both IRC and HTTP Botnet is the use of central Command and Control. This characteristic provides the attacker with very well-organized communication. However, the assets also considers as a main disadvantage to the attacker 8. The threat of the Botnet can be decreased and possibly omitted if the central CC is taken over or taken down 8. The method that is starting to come out is P2P structure for Botnet interaction. There is not any centralized centre for P2P botnets. Any nodes in P2P botnet behave as client and server as well. If any point in the network is shut down the botnet still can continue its operation.The storm botnet is one of the main and recognized recent P2P botnets. It customized the overnet P2P file-sharing application which is based on the Kademlia distributed hash table algorithm 55 and exploit it for its CC infrastructure. Recently many researchers specially in the anti-virus community and electronic media concentrated on storm worm 56,57.2 .2 Background and HistoryA peer-to-peer network is a network of computers that any computer in the network can behave as both a client and a server.Some explanation of peer-to-peer networks does not need any form of centralized coordination. This definition is more comfortable because the attacker may be interested in hybrid architectures 8.2.2.1 HistoryThe table 2.1 shows a summary of some well-known bots and P2P protocols. The range of time from the first bots, EggDrop, until the Storm Worm P2P bot is newly released. The first non-malicious bot was EggDrop that came up many years ago, and we know it as one of the first IRC bots that came to market. GTBot that have many other categories is another well-known malicious bot, that its variants are IRC client, mIRC.exe61.After a while, P2P protocols have been used for Botnet activities. Napster is one of the first bot that used P2P as its communication. Napster built an platform that permit all bots can find each other and share files with each other in the network. In this bot, file sharing has been done in the centralized server that we can say it was not completely a P2P botnet. Therefore, all bots have to upload an index of their files to the centralized server and also if they are looking for other files among all bots, have to search in centralized server. If it can find any file that looking for, then can directly connect to that bot and download what they want. Nowadays, because Napster has been shutdown as their service recognized as illegal service, many other P2P service focusing on avoiding such finding.After few years after Napster, Gnutella protocol came up as the first completely P2P services. Actually after Gnutellas , as shown in Table 2.1, many other P2P protocols have been released, such as Kademilia and Chord. This two new p2p service are using distributed hash table as a method for finding information in the peer-to-peer networks.Agobot is another malicious P2P bot that came up recently and b ecome widespread because of good design and modular code base 61. Nowadays many researchers are concentrating on P2P bots and there is an anticipation that P2P bots will reach to the stage that Centralized botnets will not been used any more in the future.Table 2.1 P2P based Botnets2.3 Peers-to-Peer Overlay NetworksOverlay networks are categorized into two categories Structured and Unstructured. All nodes in first category can connect to most X peers regarding some conditions for identification of nodes that those peers want to connect. However in unstructured type there is not any specified limit for the number of peers that they can connect, in spite of the fact that there is not any condition for connecting to other peers. Overnet is a good example of structured p2p networks and Chorf is a good example of unstructured P2P networks.2.3.1 Brief overview of OvernetOne of the popular file sharing networks is Overnet that use for their design use distributed hash table (DHT) algorithm that called Kademlia55. Each node produces a 128-bit id for joining the network and also use for sending to other node for introducing itself. Actually each node in the network saves the information about other nodes in order to route query messages.2.3.2 Brief overview of GnutellaGnutellas is a unstructured file sharing network. In this network, when a node like n want to connect to a node like m, use a ping message to inform the other node for its presence. As long as node m received ping message, then send it back to other nodes in its neighbor and also send a Pong message to the sender of ping message that was node n. this transaction among node let them to learn about each other.2.4 Botnet DetectionIn particular, to compare existing botnet detection techniques, different methods are described and then disadvantages of each method are mentioned respectively.2.4.1 Honeypot-based trackingHoneypot can be used to collect bots for analyzing its behavior and signatures and also for t racking botnets. But using honeypots have several limitations. The most important limitation is because of limited scale of exploited activities that can track. And also it cannot capture the bots that use the method of propagation other than scanning, such as spam. And finally it can only give report for infection machines that are anticipated and put in the network as trap system. So it means that it can not give a report for those computers that are infected with bot in the network but are not devoted as trap machines. So we can come to this conclusion that generally in this technique we have to wait until one bot in the network infect our system and then we can track or analyze the machine.2.4.2 Intrusion detection systemsIntrusion detection techniques can be categorized into two categories host-based and network-based solution. Host-based techniques are used for recognizing malware binaries such as viruses. A good example of this type is anti-virus detection systems. However, w e know that anti-virus are good for just virus detection. The most important disadvantages of anti-virus are that bots can easily evade the detection technique by changing their signatures easily, because the detection system cannot update their databases consistency. And also bots can disable any anti-virus tools in the system to protect themselves from detection.Network- based intrusion detection system is another method for detection that is used in the field of botnet detection. Snort67 and Bro68 are the two well-known signature based detection system that are used currently. They use a database as signatures of famous malicious activities to detect botnets or any other malware. Actually if our objective is using this technique for botnet detection, we have to keep updating the database and recognizing all malware quickly to make a signature of it and add to our database. For solving this solving this problem recently researchers are using anomaly based IDS that can detect malic ious activities based on behavior of malware or detection techniques.2.4.3 Bothunter Dialog correlation-based Botnet detectionThis technique developed an evidence-trail approach for detecting successful bot infection with patterns during communication for infection process. In this strategy, bot infection pattern are modeled to use for recognizing the whole process of infection of botnet in the network. All behavior that occur the bot infection such as target scanning, CC establishment, binary downloading and outbound propagation have to model by this method. This method gathers an evidence-trail of connected infection process for each internal machine and then tries to look for a threshold combination of sequences that will convince the condition for bot infection 32.The BotHunter use snort with adding two anomaly-detection components to it that are SLADE (Statistical payLoad Anomaly Detection Engine) and SCADE (Statistical scan Anomaly Detection Engine). SCADE produce internal an d external scan detection warnings that are weighted for criticality toward malware scanning patterns. SLADE perform a byte-distribution payload anomaly detection of incoming packets, providing a matching non-signature approach in inbound exploit detection 32 .Slade use an n-gram payload examination of traffics that have typical malware intrusions. SCADE execute some port scan analysis for incoming and outgoing traffics. Actually BotHunter has a link between scan and alarm intrusion that shows a host has been infected. When a adequate sequence of alerts is established to match BotHunters infection dialog model, a comprehensive report is created to get all the related events participants that have a rule in infection dialog 32. This method provides some important featuresi. This technique concentrates on malware detection by IDS-driven dialog correlation. This model shows an essential network processes that occur during a successful bot infection.ii. This technique has one IDS-indepe ndent dialog correlation engine and three bot-specific sensors. This technique can automatically produce a report of whole detection of bot, as well as the infection of agent, identification of the computer that has been infected and source of Command and Control centre.2.4.3.1 Bot infection sequencesActually understanding bot infection life processes is a challenging work for protection of network in the future. The major work in this area is differentiating between successful bot infection and background exploit attempt. For reaching to this point analysis of two-way dialog flow between internal hosts and external hosts (internet) is needed. In a good design network which uses filtering at gateway, the threats of direct exploitations are limited. However, contemporary malware families are highly flexible in their ability to attack vulnerable hosts through email attachments, infected P2P media, and drive-by download infections 32.2.4.3.2 Modeling the infection dialog processThe bot distribution model can conclude by an analysis of external communication traffics that shows the behavior of relevant botnet. Incoming scan and utilize alarms are not enough to state a winning malware infection, as are assumed that a stable stream of scan and exploit signals will be observed from the way out monitor 32.Figure 2.1 shows the process of bot infection in BotHunter that used for evaluating network flows through eight stages. This model is almost similar with the model that Rajab et al. presented for IRC detection model. The model that they proposed has early initial scanning that is a preceding consideration happen in form of IP exchange and pointing vulnerable ports. Actually figure 2.1 is not aimed for a strict ordering of infection events that happen during bot infection.The important issue here is that bot dialog processes analysis have to be strong to the absence of some dialog events and must not need strong sequencing on the order in bound dialog is conducted. On e solution to solve the problem of sequence order and event is to use a weighted event threshold system that take smallest essential sparse sequences of events under which bot profile statement can be initiated 32. For instance, it is possible put weighting and threshold system for the look of each event in a way that a smallest set of event is important prior of bot detection.2.4.3.3 Design and implementationMore attention devoted for designing a passive network monitoring system in this part which be able of identifying the bidirectional warning signs when internal hosts are infected with b