Becoming an Internet Service Provider

by Rob Kolstad, Berkeley Software Design, Inc.

Becoming an Internet Service Provider is not a mysterious or difficult process. This document outlines the steps necessary.

This is version 0.9.1 of a brand-new document. Every effort has been made to ensure accuracy; no promises are made. Please communicate any errors to the author, [email protected], +1 719-593-9445.


For a topic that has seen so much hype of late, it is amazing how little is explained about what the Internet is and how one leverages it. This white paper gives a brief explanation of the Internet, services it provides, and how new `Internet Service Providers' start up their operation.

This article is written as an overview. It is not intended to be 100% complete (that would take an entire book!). Finally, it is written from the biased point of view of someone who does not wish to spend tens of thousands of dollars to start such a service. This bias extends to the occasional emphasis on BSDI's products.

The Internet: A Brief Overview

The formal definition of the Internet says ``The Internet is a network that connects thousands of other computer networks''. This does not seem to be particularly helpful in understanding what's really going on.

Computer networks became practical when Ethernet and the Berkeley 4.1 follow-on releases of the Berkeley BSD UNIX system started becoming widely available back in the 1982 timeframe. These `Local Area Networks' (LANs) typically spanned an area smaller than a couple square miles. Some hardware existed to connect these LANs to extend the area slightly, but not on a nation-wide scale.

Prior to and concurrently with the development of these small high speed networks, the ARPA (US government research agency) folks were funding research on the `ARPANet', a lower speed (64,000 bits/second and slower) network that connected a few dozen highly distributed institutions (using a cross-country communications backbone). These were the days when 56,000-64,000 bits/second was an incredibly high speed that was astounding in its transfer capacity.

In the 10-12 years after 1982, local Ethernet (and, later, token ring) networks penetrated the marketplace in large companies and institutions, educational institutions, and, ultimately, smaller organizations. Ethernet hardware can now be had for less than US$50/node.

Users of local networks exploited them by sharing files (much easier than moving floppy disks or tapes from one computer to another), communication (e.g., electronic mail), remote printing (i.e., sharing a printer among several workstations or PCs) and occasionally other features (e.g., remote job execution). These types of communication, of course, are desirable on a larger scale, as well.

While Ethernet was gaining its foothold, another kind of communication evolved in the UNIX world as groups leveraged a program called `UUCP' (``Unix to Unix CoPy'') to share electronic mail and newsgroups (which are discussions forums somewhat similar to bulletin boards) via a loosely organized nation-wide dialup `network' called USENET. Messages were passed from computer to computer with a routing scheme that often required the sender of a message to know the exact path of 10 computers that would process a message before it was delivered to its recipient. This nation-wide network whetted the appetite of the technical community for ever-larger and ever-faster networks. E-mail delivery on USENET could take hours or days, depending on the frequency of telephone dialing at the various sites.

As communications technology costs decreased, computers with LANs proliferated, and the TCP/IP protocol was proven successful, the notion of a nation-wide or world-wide network continued to assert itself in the minds of many.

This network would provide communication 24 hours/day with permanent connections between large numbers of computers. Transferring small packets of information one at a time, communication across large distances required only a fraction of a second, since all the communication lines were always connected. A packet from New York City to San Francisco might pass through eight different `routers' on its way to its destination - but each `hop' requires only a few thousandths of a second, thus effecting a near instantaneous cross-USA connection.

The Internet is the term coined to describe the interconnection network that facilitates communication among all the smaller networks and individual computer systems that connect to it.

With the dramatically increased bandwidth of fiber, decreased digital communication costs, and a number of high-tech firms marketing and selling products, the Internet has taken on a life of its own.

Connecting to the Internet

Interestingly enough, the Internet's connectivity aspects are not centrally organized. For instance, BSDI's headquarters uses one provider who has connection points in certain major cities. The BSDI Colorado Springs office has a direct telephone-company supplied line to Dallas, where BSDI's data joins other firms' data in a sort of information pipeline to a bigger backbone, which ultimately delivers the packets across the world.

This lack of centralization means that the Internet is not organized like an army with a `President' node at the top with `General' nodes directly beneath it. The Internet is more like a hodge-podge of various interconnections that resemble more a crazy game of connect-the-dots than a cleverly designed backbone-with-branches. This is not a criticism! Each node was connected to the Internet in a perfectly rational way - but typically by an organization that had specific goals for its use and its customers' use. These strange connections make more sense when viewed in the light of the USA's communications tariffs that sometimes favor longer hops between states over shorter hops within a state.

Many people wish to connect `directly to the Internet'. This connotes an idea that there's some `backbone' that affords a `direct' connection, similar to the way a water company might run a water main directly down a major street of a town. This is not the way it works, though.

To connect to the Internet, one identifies an `Internet Service Provider' (ISP) that is already connected to the Internet and negotiates a business agreement to join the Internet through them. The list of ISPs is large and includes tiny ISPs with a single computer and some dial-in lines and large ISPs with thousands of miles of fiber strung around the country. Different ISPs offer different strengths and different costs. To obtain a mostly complete list of service providers that serve your USA areacode, send electronic mail to isp-r[email protected] that includes, in the body, a line or two like:

and you will be sent back a list of local and national providers for the areacodes you list.

Internet Services: A Brief Overview

The world of the Internet Service Provider is an interesting one because it includes both halves of the client-server pair. On the one hand, clients of an ISP might dial in to the ISP's machine and access Internet services from there. On the other hand, users from across the Internet might wish to view a World Wide Web page of a merchant that the ISP supports.

The various services an ISP might offer or exploit include (in no particular order):

Obviously, communication of text and graphics is the centerpiece of the various Internet services. The hype in the media through 1994 and 1995 can help fire your imagination for all the ways to exploit these services.

Communications Protocols

Just as humans use certain rules and schemes for communications, so do computers. Interestingly enough, the world of 1's and 0's includes many different ways to communicate information. Each of these ways is called a `Communications Protocol' or just `Protocol'. The Internet uses the TCP/IP protocol. Sometimes other protocols are `encapsulated' in the TCP/IP protocol to be moved from one point to another, but generally TCP/IP is the lingua franca of the Internet.

When using TCP/IP on telephone lines, one of two schemes is used: SLIP or PPP. These protocols frame the TCP/IP packets correctly onto the phone line and deliver them to the other end. PPP can also transport other protocols, but that won't be discussed here.

Becoming an Internet Service Provider

Internet Service Providers require a number of components to succeed:

Finding an ISP

Finding the organization to provide Internet connections is easy (contact BSDI and tell them your areacode; see the e-mail request scheme above).

Procuring a Communications Line

Procuring a communications line is a bit more difficult. Happily, your service provider will help you do so. Your site must have a wire to connect to your own provider. The local phone company almost always owns that part of the proposition. They run a `local loop' from your site to the phone company central office. From there, your provider might have a `Point of Presence' (POP - but not to be confused with the POP mail protocol) to which your wire connects. If not, your wire will be routed either through another `local loop' to the provider or via a `long haul line' (long distance) to the provider. The `local loop' charges are often the highest charges in the communications chain.

These days, many sites have several options for connecting to their ISP (and for your own customers to connect to you). These include:

More exotic expensive high speed protocols like ATM are not discussed here.

The tiniest Internet service consumers can get great leverage from voice-grade modems. The telephone line is inexpensive; the connection hardware is widely available. The speeds, however, are not that high. The best bet in this category is the 28,800 bps modem. With compression, these modems can occasionally achieve more than 40,000 bits per second - a substantial fraction of the dramatically more expensive 56K or Frame Relay lines. Communications on voice-grade lines (both into and out of your Internet server) pretty much require the use of the PPP or SLIP protocols. Both ends of the communication link must support these protocols.

You might wish to use the higher speed 56K or T-1 lines. These are discussed in the same paragraph because their only differences are speed and cost. The 56K lines run at 56,000 bits per second; T-1 lines run in excess of 1,500,000 bits per second. Protocols that use these lines are `serial' and include, among others, the cisco-HDLC protocol and the PPP protocol. The phone company supplies your line; you or your service provider supplies the CSU/DSU (a box that looks like a modem but isn't a modem). You or your service provider supplies a `router' that transfers packets from the line into your computer or network and vice-versa (more on this later).

ISDN (`Integrated Services Data Network') is gaining much publicity of late since it is finally being deployed after a decade of announcements. ISDN lines (e.g., for consumers) each supply two B-channels and one D-channel for a total of 120+ kilobits/second when properly driven. The issues with ISDN in the first part of 1995 in the USA include:

ISDN's speed is very good. ISDN's connectivity, as seen above, must be studied carefully. ISDN can be a fantastic way for connecting to your own ISP. Connecting each user to your computer using ISDN might be too expensive.

ISDN has a different set of features that are also attractive. Because ISDN has two channels, it can support up to two voice-grade signals per `ISDN wire'. Some firms sell a device which can de-multiplex those signals into a modem so that you can get two incoming dialin lines for the price of one ISDN line. This can be a very cost-effective proposition. I don't have the names of these vendors at hand right now.

Figure 1: Expensive ISP Configuration

Figure 2: Less Expensive ISP Configuration

Frame Relay is another connection scheme that, like the 56K and T-1 links, uses a serial bit stream. Many local telephone companies (like USWest) are constructing large `Frame Relay Clouds'. (Many network designers draw wide-area-network transport layers as a `cloud' into which they connect their various hubs.) In Colorado, we are trying to exploit the pair of state-wide Frame Relay Clouds to connect all the public schools to the Internet. The advantage of the Frame Relay Cloud is that connections are priced at a fixed (low) cost per month and a `Permanent Virtual Circuit' links you to any other one site - anywhere within the cloud - for the same cost. This distance-insensitive pricing can yield attractive benefits in certain situations. Frame Relay connections require both the Frame Relay protocol and serial line interface hardware.

        |  Scheme      Speed (kbits/sec)   Relative Cost |
        |Modems            14.4-28.8            Low      |
        |56K lines            56              Medium     |
        |ISDN                120+           Low-Medium   |
        |T-1                1,500+             High      |
        |Frame Relay          56            Low-Medium   |
        |Frame Relay        1,500+          Medium-High  |

Procuring ISP Hardware and Software

To be an ISP, you need a certain amount of hardware:

Figure 1 illustrates one way of creating a small ISP site. The cost of the router, however, can be quite high (US$1,000 - US$2,500 or more). Figure 2 shows a site with an integrated router/Internet-server and two different schemes for connecting modem users. In Figure 2, no Ethernet is required if there are no other entities to connect.

Many people ask if the CSU/DSU can be eliminated (it costs a few hundred dollars). If you're using a 56K/T-1 line (including Frame Relay), then the CSU/DSU is pretty much required. Some new interface cards might be released in 1995 that include the CSU/DSU in the interface card.

Figure 2 has a trick in it - there's a high speed serial interface card inside the Internet server. These cards (different ones for straight 56K/T-1 or Frame Relay) cost about US$600.

Hardware Checklist

Here's a checklist for procuring hardware for your ISP system if you intend to use the BSDI-based Internet Gateway software:

About using SCSI

Installing a SCSI adaptor ensures that your system will be expandable in the future and have access to a very wide variety of commodity peripherals. Each SCSI adaptor enables connection of up to seven peripherals. This means easily-expandable disk, tape, and other SCSI peripherals.

Bleeding-edge Hardware

I am often asked ``How can I configure the fastest possible server for my users?'' In 1995, the interesting answer is: Get a faster network link. The 90 MHz Pentiums now exhibit 90 MIPS (the 90 number is entirely coincidental!) on my benchmarks. That's almost the same as a Sun SparcStation-10 (though the Sun has faster floating point calculations for engineering CAD computation). On my benchmarks, integer performance is 2.5x faster than an RS6000 model 550, and 1.6x faster than a PowerPC. The formerly top-of-the-line DEC 200MHz Alpha processor was about 20% faster on integer performance; the higher speed Alpha processors should be commensurately faster.

These numbers are startling when compared to the marketing hype afforded all these various systems. They are also startling in light of the abysmal performance given by DOS, Windows, and OS/2 on PCs. BSDI's disk buffering and efficient memory management algorithms can make your PC appear to run 10x faster than it did under these other operating systems!

In summary, a 90 MHz Pentium is one of the most powerful processors ever marketed and is more than adequate to support literally hundreds of simultaneous users.

How Many Users?

Compuserve benchmarked their 486/66 file downloading tools with 260 simultaneous users (Pentium-90 is 2.5x faster than a 486/66). This benchmark used Ethernet rather than actual teletype lines for communications.

Can your system support 100 users or more? It depends on their activity. If they are all recompiling 10,000 line C programs, then the Pentium-90 is far less entertaining with five users on it than with one (since the processor then appears to be only 20% of what it was). On the other hand, your users are probably reading mail, newsgroups, and transferring files. In this case, communications capacity is the key.

Users that utilize local services are limited by their modem or network communication speed. Reading mail, reading news, or sending mail/news are activities that use low-medium bandwidth in a bursty fashion. Even 100 users downloading files continually is only 28,800 bits/second/user x 1.7 (compression) x 100 users = 4,896,000 bps = 612,000 bytes/second, less than one Ethernet. BSDI's system can, in fact, saturate Ethernets to their maximum capacity (about twice the previous 612KB/sec).

Note, however, that if the 100 users above are accessing Internet facilities (e.g., by using WWW), then they can be bottlenecked severely by the network bandwidth. A 56K line is only 7,000 bytes (not bits) per second. That's 1% of the capacity listed above. A T-1 is faster: just under 200,000 bytes/second. You can figure out your users' consumption rate to see how many bytes/second they require. I always figure about 10,000-30,000 bytes per WWW picture. If all the users are loading various WWW graphics simultaneously, they could end up waiting a while even on a T-1.

BSDI does not yet support dual-processor Pentium systems, by the way.

Connecting Other Users

Users connect to your system either through modems, directly connected serial lines, or Ethernet (using the telnet or X windows protocols). BSDI does not currently supply a scheme that enables Novell users to access the server using IPX/SPX - Novell sites will need to acquire the TCP/IP option for their Novell servers.

If you have more than two modems, you probably wish to purchase a multi-port serial board. These boards enable connection of large number of serial devices (e.g., modems) to the system.

Internal Router Software vs. External Routers

Instead of buying an external router for US$1000-US$2,500, BSDI's system enables you to use the N-2 card from SDL Communications (maybe US$600; call 508-238-4490 for details) for 56K/T-1 cisco-HDLC or PPP communications - or the Frame Relay interface card from Emerging Technologies (call 516-271-4525) for Frame Relay communications. Check with your ISP bandwidth provider to ensure the protocols are compatible.

External routers are, of course, usually PCs with routing software (just like the BSDI Gateway is). They are often small 386s, since routing takes very little CPU time. Once configured, they typically require 0 maintenance and 0 configuration at reboots and in ongoing operation.

Integrated routers (a la BSDI) share these same traits.

How Long Does It Take?

Getting an ISP business off the ground can take a short while or a long one, depending on circumstances. The potential bottlenecks are:

Check the communications install times first. Hardware is a 1.5 week proposition from many suppliers. Software is two-days or sooner from BSDI. Configuration and testing can take from 2 hours to 2 weeks, depending on luck and skill.

How Much Does It Cost?

Costs vary across a spectrum, of course, depending on requirements. Those of us who have been purchasing computers for more than just a couple years are pleased and astonished by the incredible economy of the components. On the other hand, those who have only dealt in the sub-$1,000 market think these prices are fairly high. They're reasonable - and get cheaper every few months.

Hardware costs (all in US dollars and estimated - no promises, just the best guesses we can come up with):

So, excluding modems, a 32-user system (40 MB RAM plus 1 GB disk) should be had for ballpark of US$10,000. Modems could run another $6,400. Installation for the phone lines is also a big financial hit, potentially more than US$100 per phone line.

Software runs from $500 to $3,000, depending on vendors, discounts, and the like. Call BSDI!

Communications costs will run $50 to $3,000 per month, depending on speed. 56K lines are closer to $200-$500/month.

How do you choose a provder for your own Internet access? How come the providers have such wildly differing prices? Here is some insight on choosing a provider:

BSDI Technical Details - Questions & Answers

How do I account for users' activities?
BSDI's accounting includes connect time (per user), CPU seconds (per user and per process), disk usage (with disk quotas), and complete WWW accounting (hits/page along with complete list of which users are accessing your server's pages).

Can I host multiple domain names on a single computer?
Yes, you can target as many domain names to one system as you wish. Likewise, a single system can have multiple IP addresses. There is, however, a mitigating factor. When sending mail from a system with multiple domain names, the system administrator must create a rule to specify which domain name is shown as the return address.

Can I host WWW pages for multiple domains/names?

How do I get domain names and an IP address?
The Network Information Center (NIC) is the central administration authority for names and IP addresses. You can fill out one of their forms and submit it (electronically, catch-22) or have your ISP do it for you (the usual case).

What about documentation? I need help administering my system and the network services.
Evi Nemeth, et al.'s Unix System Administration Handbook addresses BSDI administration in great detail in its second edition. Prentice-Hall publishes that one.

O'Reilly and Associates reprints the entire 4.4BSD manual set in five volumes.

O'Reilly and Associates also publishes Managing Internet Information Services by Liu, et al. Contact O'Reilly at 800-889-8969.

Of course, BSDI supplies installation documentation on paper and loads of technical information on its CDROMs in addition to its 800 number support line. BSDI does not, however, offer system administration training as part of its 800-number support system.

I'm worried about my system's security. What should I do?
First of all, be sure you have something to worry about. If you're running a public-access bulletin board system, you don't care if people steal all your data (it's public, anyway). You do care if people can `hack' your machine and change information or deny service, of course.

BSDI's systems have a sterling reputation in the security community. BSDI strives to maintain this reputation by fixing bugs as soon as they are reported.

My management says that they won't connect our system to the Internet where college students can hack our MIS database.
Your management is being prudent! You might be able to gain great leverage from a `firewall'. A firewall system sits between your network and the Internet and monitors every packet that goes by. Packets that do not meet specific security requirements are discarded. This keeps `hackers' from entering your internal network and wreaking havoc. The ftp site has a set of security tools to construct a firewall using a BSD/OS system. Recent experiences show that it takes about two person-days to construct and configure a firewall using this technique.

I see BSDI sells source code. Do I need source code?
Most customers do not require source code. Those who desire it cite several reasons: they are not dependent on anyone when they run into an emergency, they can modify the code to meet their specific advanced requirements, and they feel `warm and fuzzy' when they have the source code.

My ISP is charging me 10x as much as my neighbor for network bandwidth. Why is he cheating me?
He's probably not cheating you. If your neighbor is a small business or non-ISP business, he's probably using only a tiny fraction of the network bandwidth capacity afforded him. Your bandwidth provider is charging you for the resources you'll be using, particularly the valuable pipeline to the Internet. It is important, of course, for you to notify your bandwidth supplier that you are reselling the bandwidth. Otherwise, you might find yourself with only 10% of what you had anticipated.

What software do the customers use on their computer?
Your customers will either dialin to your computer using ProComm any other ASCII modem/dialin program or they will use SLIP or PPP to establish a full TCP/IP connection. BSDI is negotiating with another vendor that offers TCP/IP-like connectivity in a serial-line environment but has not yet completed those negotiations.

ASCII dialin users typically see a `glass teletype', often with 24 lines of 80 characters each. They use the dialin/modem program to access megabytes of netnews in addition to exchanging personal e-mail, chatting in hundreds of `chat groups' on IRC, and surfing the World Wide Web via `lynx' - the non-graphic WWW interface. Since it has no pictures (and concomitant transfer overhead), lynx offers much snappier performance than full-blown graphic World Wide Web browsers. Lynx is also reputed to have a `gopher' interface.

ASCII dialin users can also move files with a combination of ftp (from the Internet to the ISP machine) and then zmodem to their machine.

Customers who want full TCP/IP connectivity (which is pretty much required for graphics) need a SLIP or PPP software package. Internet-in-a-box is a typical product that offers SLIP or PPP connectivity. Once they have connectivity, they can exploit a full graphic WWW/Mosaic browser (e.g., transferred from the ftp archives of NCSA at University of Illinois), NCSA Telnet to login to other systems (also available from ftp archives at UI), and NCSA ftp to move files from other systems (available likewise). They'll probably use either ASCII dailup or NCSA telnet to read mail and netnews on your machine - so TCP/IP connectivity doesn't fully relieve your machine of providing services besides `routing'. The NCSA programs are available, to the best of my knowledge, for both Macintoshes and PCs and are provided with no charges.

Some newer products and offerings enable voice or even video to be transmitted in real-time across the net. These products will saturate your network link's bandwidth so quickly as to be amazing. BSDI is still studying the impact of these products and does not yet recommend their use other than in the context of a technology demonstration.

How much impact is a newsfeed on my system?
As of early 1995, A UUNET-sized ``full'' newsfeed is ~120K articles/day, with about 300MB of data for 100% of the articles (not all sites collect 100% of the articles). The folks over at UUNET, perhaps the USA's largest feeder of news, use a 48MB Pentium-90 with 3GB of disk to feed 30 sites (i.e., 3,600,000 articles a day, total of 9GB of newsfeed per day).

A 128MB Pentium-90 with extra disk buffers can feed over 40 sites per day.

Conclusion? A big newsfeed takes only a small percentage of your machine, presuming that you have enough RAM and disk I/O bandwidth available.