This article is part 1 of 2. In this article I will be illustrating how to use members of the System.Net namespace to create a business object that "Gets" all news headlines that ASPWire has available, parses through the return, constructs a DataSet object that holds a DataTable of all news items, and finally returns a DataView that you can bind to. We also make use of Page Caching to limit our request for the data to 3 times per day. We cannot give all the code necessary for you to run this example on your own because if we did I would get banned from ASPWire, but for those of you who already promote ASPWire on your site you will easily be able to figure it out. All you need to do is replace the fake URL in the code example with your personal URL. For those of you who have no idea what I am talking about, you will be able to get the general idea of what I am doing by the code example. For a background see the following section of the article
Background
ASPWire is a news and information source that many sites use to give their users up to the minute information on new products, tutorials, free stuff, and much more. Some of you may have even come to our site from an ASPWire News link. ASPWire offers an affiliate program to sites like ours so we can help evangelize this cutting edge news. They offer many ways to get this news, one is to use their pre-made java script files that dynamically creates a list of headlines on your site; this is great for some people, but if you want to maintain a design flow on your site you have to follow their color scheme. A way to get around this is to access their headlines programmatically using some kind of HTTP GET component. Using this method you get to make the headlines follow the same design scheme already in place on your site. The return from the GET comes in a file format that is easy to parse; each headline is broken up into 4 lines i.e. line one might be the URL to the first article, line 2 would be the date the article was released, line 3 would be what type of news it is, and Line 4 the headline. Line 5 would be the next article URL and so on. One stipulation on using this type of access is you can request the page more than 4 times per day
Let's Get Started
I first will be going over the objects used in the code example, then I'll give the code example, and finally I'll quickly walk through how the code works. For a working example there is a link provided in the article.
The first object I want to tell you about is the WebRequest class, found in the System.Net namespace. The WebRequest class is the base class that applications use the Request and Response mode for accessing data from the Internet. You can use this class to hold an instance of a WebRequest object. To return a WebRequest object you can use the WebRequestFactory object. The WebRequestFactory is a static class that returns an instance of an object derived from the WebRequest Object based on a Uniform Resource Identifier (URI), or URL passed into it's Create method. Once you have the WebRequest object you can call the GetResponse Method of the WebRequest class; which returns the actual response data to the request made in the WebRequestFactory.Create method. One way to make use of the data returned is to create a StreamReader object, constructing it by passing in the return of the WebRequest's GetRequestStream method, which returns a stream of the data returned by the request, this can be Html, Raw Data, etc.
Well enough background, let's look at the code! Code Listing 1.1 is the class file for this project. The code in this project is broken up into three sections. First is the VB class file, second is the make file for the class file, and finally the Web Form I use to show the results. Again, the URI that I pass into the WebRequestFactory will not be included, but with minor changes to the code you can implement this for your own purposes, or you can make a file that is constructed the same way as described in the Background section of this article.
Code Listing 1.1
[VB Class File]
Imports System
Imports System.Net
Imports System.IO
Imports System.Data
Imports System.Collections
Namespace ASPWireUtility
Public Class ASPWireGetNews
Public Function Main(ByVal IN_URL As System.String, ByVal IN_VALUE As System.Integer) As DataView
'Create WebRequest Object
'------------------------
Dim webReq As System.Net.WebRequest
webReq = System.Net.WebRequestFactory.Create(IN_URL)
'Create WebRespone Object
'------------------------
Dim theReturn As System.Net.WebResponse
theReturn = webReq.GetResponse()
'Create Stream Object
'--------------------
Dim theStream As System.IO.StreamReader
theStream = New StreamReader(theReturn.GetResponseStream(), System.Text.Encoding.ASCII)
'Create DataTable
'----------------
Dim dt As DataTable
dt = New DataTable("ASPWire")
'Create ArrayList to hold all columnNames
'----------------------------------------
Dim columnNames As New ArrayList()
With columnNames
.Add("URL")
.Add("Date")
.Add("Type")
.Add("HeadLine")
End With
Dim myEnumerater As System.Collections.IEnumerator
myEnumerater = columnNames.GetEnumerator
While (myEnumerater.MoveNext)
'Create Columns
dt.Columns.Add(MyColumns(myEnumerater.Current.ToString()))
End While
'Create DataSet object
'---------------------
Dim ds As New DataSet()
'Add The DataTable To The DataSet
'--------------------------------
ds.Tables.Add(dt)
'Add Rows To The Table
'---------------------
Dim i As System.Integer
For i = 0 To IN_VALUE - 1
Dim dr As DataRow
dr = dt.NewRow()
dr("URL") = theStream.ReadLine
dr("Date") = theStream.ReadLine
dr("Type") = theStream.ReadLine
dr("HeadLine") = theStream.ReadLine
dt.Rows.Add(dr)
Next i
'Return DataView
'---------------
Return ds.Tables(0).DefaultView
End Function
Public Function MyColumns(ByVal strColumn As System.String) As System.Data.DataColumn
'Function To Create Columns
'--------------------------
Dim dc As DataColumn
dc = New DataColumn()
With dc
.DataType = System.Type.GetType("System.String")
.AllowNull = True
.Caption = strColumn
.ColumnName = strColumn
.DefaultValue = " "
End With
Return dc
End Function
End Class
End Namespace
[MAKE FILE - MAKE.BAT]
vbc /t:library /out:PathToFile\ASPWIRENEWS.dll PathToFile\ASPWIRENEWS.vb /r:System.IO.dll /r:System.Net.dll /r:System.dll /r:System.Data.dll
[Web Form]
<%@ OutputCache Duration="28800" %>
<%@ Import Namespace="ASPWireUtility" %>
<html>
<head>
<script runat="server" language="vb" >
Sub Page_Load(SRC As System.Object, _
E As System.EventArgs)
Dim wr As New ASPWireGetNews
DG.DataSource = wr.Main(Your URL, 10)
Page.DataBind
Response.Write(DateTime.Now.ToString())
End Sub
</script>
</head>
<body>
<form method="post" runat="server">
<asp:DataGrid
id="DG"
runat="server"
AutoGenerateColumns="False"
GridLines="None"
ItemStyle-BackColor="#EEEEEE"
ItemStyle-ForeColor="#000000"
AlternatingItemStyle-BackColor="#FFFFFF"
AlternatingItemStyle-ForeColor="#000000"
>
<property name="columns">
<asp:TemplateColumn HeaderText="Unit Price">
<Template name="HeaderTemplate">
<h3><a href="www.aspwire.com">ASPWire News</a></h3>
</Template>
<Template name="ItemTemplate">
<table cellpadding="4" cellspacing="0" border="0">
<tr>
<td>
<a href="<%# DataBinder.Eval(Container.DataItem, "URL") %>">
<%# DataBinder.Eval(Container.DataItem, "HeadLine") %>
</a>
</td>
<td>
<%# DataBinder.Eval(Container.DataItem, "Type") %>
</td>
<td>
<%# DataBinder.Eval(Container.DataItem, "Date", "{0:c}") %>
</td>
</tr>
</table>
</Template>
<Template name="FooterTemplate">
</Template>
</asp:TemplateColumn>
</property>
</asp:DataGrid>
</form>
</body>
</html>
Break It Down
First I'll go through the VB Class file. There are a total of two methods in this class file; the Main function, which returns a DataView and the MyColumns function that returns a DataColumn. The MyColumns function is used by the Main function; I'll be going over that a little later.
The first part of the main function has two parameters, the IN_URL and the IN_VALUE. The IN_URL is the URL that I use to create the WebRequest object, and the IN_VALUE is an integer that specifies how many headlines to return. The maximum number of news items available at any one time is 20. The first part of the Main function is the creation of our WebRequest object and putting it into a Stream object so I can read through it. The next part of the function is the construction of the DataSet. The first thing I do is create a DataTable object, next I create an ArrayList object that I use to construct the columns of the new table. The reason I created an ArrayList to create the columns is because I wanted to make the actual creation of the Columns collection as easy and extensible as possible. Presently, there are only four rows returned in our GET, but in the future there may be five or six. By creating the columns in this fashion I need only call another add method of the ArrayList passing in the new column name to create an additional column. After I create the ArrayList I loop through all the elements calling the Add method of the DataTable.Columns collection while passing in the result of the MyColumns function. The MyColumns function expects one parameter, a String; which will be the name of the new column. Once the loop is finished I create a new DataSet, and add the DataTable created to the DataSet. Next I add Rows to the now created DataTable, I do this by doing a For Loop, starting at 1 ending with the value specified in the IN_VALUE parameter. After the loop I return the Default DataView of the now fully populated DataSet.
The Make file is pretty self-explanatory. So I wont go into much detail, you have to obviously replace the paths to the files accordingly and it should compile without a problem.
The first thing I do in the Web Form is set the Output Cache to 28800 seconds, exactly 8 hours. Which means that every 8 hours the page will refresh and get a fresh copy of all articles from ASPWire; I placed a Label control on top of the DataGrid and set it's Text attribute to the time the results were last cached.
The next thing is to import the newly created namespace. Once imported you can now create the ASPWireGetNews object. After creating it you can either bind a control directly to the Main method since it returns a DataView, or you can access the rows programmatically and do what you want with the data. I chose just binding it to a DataGrid for simplicity sake. Of course we added a little pizzazz to it by applying alternating item colors and the use of the TemplateColumn
The Final Rendered page can be seen in Figure 1.1
Figure 1.1 Rendered Page
Working Sample
A working sample of this application can be viewed at Sample Here
Conclusion
In this article I went over a couple of very interested topics. First how to create a use a WebRequest object; second, how to read that objects data into a stream and finally how to dynamically create a DataSet object out of that data. Next week, in part two, I'll be demonstrating how to convert this into a Web Service. So stay tuned!!