A Funkaoshi Production

Building a Links Log in Textpattern

   23 July 2004, the wee hours

Update Sept 13th 2005: The changes needed to publish.php are slightly different for Textpattern 4.0.1. Read my article on upgrading to 4.0.1 for more details. (There is a very concise post on how I did things in 4.0.1 for those who think this post is too all over the place.)

As I’ve mentioned before, I wanted to create a link log in a similar manner to Jason Kottke. What I wanted to accomplish were the following things:

1. Links would be intermingled with posts. Everything would be displayed in chronological order.
2. Since I was displaying a list of links, the links should semantically be marked up as unordered lists.
3. I wanted to continue using Del.icio.us, as well as Textpattern.
4. I wanted to import all my old entries from Del.icio.us into Textpattern.

I’ve managed to accomplish all of these goals. So, here are some details for those of you who enjoy this sort of thing.

Creating A Link Entry

The simplest thing to do was figure out how to store a link entry in my Textpattern system. Although Textpattern does have a feature for saving links, I opted to store links as articles. The primary reason for doing so is that I’d like people to be able to comment on links. A fellow who goes by Sencer outlined what I thought was a good way to save links as articles on the textpattern forms.

There are a few elements to a link entry that need to be saved. The link itself, a title, a description, keywords/tags, a referrer name, and a referrer url. There may be more things you may want to save, but I thought this was enough information for my needs. I mapped things as follows: the link title gets saved in the title field, the description in the article body, the url in custom field 1, the referrer in custom field 2, the referrer url in custom field 3, and the keywords/tags in the keywords field.

I wanted the link entries to display differently then the rest of my posts, so I created a new form to display them with. I have to set the override form to this new form. The links all end up in their own section, linklog.

Currently I don’t set categories for the links, though I could do so based on the keywords/tags I saved.

Posting a New Link

I created a simple web page with a form for entering the information I described above. The form takes the information entered, and by directly accessing the Textpattern database, saves a new article. Ideally one would want some sort of API for posting to Textpattern; unfortunately, no such API exists. After posting to Textpattern, the page will send of a request to Del.icio.us with the same information. In this way, I can post to both places simultaneously. I made a bookmarklet that copies the URL and title of the page I am viewing into the html page I made.

Displaying the Links on the Page

As I mentioned earlier, the links are just articles. Since the links have their display form overridden they will display intermingled with my posts by default. The challenge at this step was to get them to display as lists. This involved turning my Textpattern install into a bigger Frankenstien then it already is.

The override form for each link entry would print the link out as part of a list—i.e. it would nest the entry in <li> tags. What each link would need to do is determine is if it was the first or last link in a block of links. If it was, then it would need to start or end the list block—i.e. output a <ul> or </ul> tag.

Since each link is posted to its own section, I simply needed tags to check if the section for the previous article or the next article were different from the current articles section. If the section of an entry that proceeded a link was not the section links belong to, then we know that we are starting a block of links. Similarly, if the section of the entry following a link was not the section links belong to, then we know we are ending a block of links.

I wrote functions to figure out the above information. One hiccup with this approach is that since we only display a finite number of articles on each page, it may be the case we need to end, or start, a block of links even though the next, or proceeding, article’s section is the link log section. Links at the top or bottom of the page may suffer from this problem. To get around this, I added an other variable to each article that said which article on the page it was. If we were printing a link, and it was the first article on a page, then we would always need to start the list it would be placed in, even though on the previous page it may be the case that there was a link entry. Similarly, if we are printing out a link and it was the last entry in a page, we would need to end the list it is placed in.

This step probably took the most time. The lack of documentation in Textpattern can make these sort of endeavours difficult. However, it is usually instructive to mess around with someone else’s code.

Importing My Old Del.icio.us Entries

The last thing I needed to do was import all my entries from Del.icio.us into Textpattern. Now, thankfully, Del.icio.us is an excellent and very open system. I found a simple curl command for grabbing all your links on the web, and used it to grab all my links. (The command is curl -u user:password http://del.icio.us/api/posts/recent?count=99999 > d
el.icio.us-backup.xml)

Now, I had a XML file full of links. Unfortunately, I don’t know how to parse files like that in PHP. I knew how to do so in Python, but didn’t know how to do database inserts in Python.

I began by trying to make a python script to generate the SQL commands for inserting all the links. My plan was to run them by hand. Simply copy and paste what the python script generated into phpMyAdmin and that would be that. But there were issues with un-escapped strings, and the fact I needed to textile the descriptions. So I scrapped that, and started writing a PHP script.

Now, instead of trying to learn how to parse XML in PHP, I decided I would use some simple regular expressions to turn the XML file into a PHP array. I then copied that array into a PHP program I wrote that would iterate through the array and perform SQL inserts. The advantage to going this route was that I could use a lot of the functions in Textpattern to do the database calls and format the information.

That script wasn’t too hard to write, and worked fine.

Conclusions

So there you have it. A poorly written account on how I spent the past day.

Apendix

I’ve decided to post some of the code I used to do all this here, since I’ve been asked for it on three different occasions now. If you make the changes outlined below, you can basically clone what I have done here.

I added two functions to the taghandlers.php file. They could be turned into plugins:

1. function if_next_section_different($atts, $thing) {
2. global $thisarticle, $pretext;
3. if ($thisarticle['article_num'] == 1) return parse($thing);
4. $posted = safe_field('posted','textpattern','id = '.$thisarticle['thisid']);
5. $thenext = getNeighbour($posted,$pretext['s'],'>');
6. return ($thisarticle['section'] != $thenext['section']) ? parse($thing) : '';
7. }
8.  
9. function if_prev_section_different($atts, $thing) {
10. global $thisarticle, $pretext, $limit;
11. if ($thisarticle['article_num'] == $limit) return parse($thing);
12. $posted = safe_field('posted','textpattern','id = '.$thisarticle['thisid']);
13. $theprev = getNeighbour($posted,$pretext['s'],'<');
14. return ($thisarticle['section'] != $theprev['section']) ? parse($thing) : '';
15. }
16. Download this code snippit /static/code/1.txt

I changed the getNeighbour() function in publish.php:

1. function getNeighbour($Posted, $s, $type)
2. {
3. $type = ($type == '>') ? '>' : '<';
4. $q = array(
5. "select ID, Title, Section, url_title, unix_timestamp(Posted) as uposted
6. from ".PFX."textpattern where Posted $type '".doSlash($Posted)."'",
7. ($s!='' && $s!='default') ? "and Section = '".doSlash($s)."'" : filterFrontPage(),
8. 'and Status=4 and Posted < now() order by Posted',
9. ($type=='<') ? 'desc' : 'asc',
10. 'limit 1'
11. );
12.  
13. $out = getRow(join(' ',$q));
14. return (is_array($out)) ? $out : '';
15. }
16. Download this code: /static/code/2.txt

I use this form template to post the links to my front page. I have to select it as an override form. Using conditionals you could probably get away with out doing that though.

1. <txp:if_article_list>
2. <txp:rsx_if_next_section_different>
3. <div class="post">
4. <ul class="interesting-link">
5. <li class="first-child">
6. <txp:else />
7. <li>
8. </txp:rsx_if_next_section_different>
9. <txp:if_custom_field name="url"><a href="<txp:custom_field name="url" />"><txp:title /></a>&nbsp;<txp:body /><txp:else /><txp:body /></txp:if_custom_field><txp:if_custom_field name="referrer"> (via&nbsp;<a href="<txp:custom_field name="referrer url" />"><txp:custom_field name="referrer" /></a>)</txp:if_custom_field>&nbsp;&nbsp;<txp:comments_invite /><txp:rsx_frontend_edit_link prefix="&nbsp;&nbsp;[" suffix="]">Edit</txp:rsx_frontend_edit_link></li>
10. <txp:rsx_if_prev_section_different>
11. </ul>
12. </div>
13. </txp:rsx_if_prev_section_different>
14. </txp:if_article_list>
15.  
16. <txp:if_individual_article>
17. <div class="post">
18. <h2><txp:if_custom_field name="url"><a href="<txp:custom_field name="url" />"><txp:title /> &rArr; </a><txp:else /><txp:permlink><txp:title /></txp:permlink></txp:if_custom_field></h2><p class="inline">&nbsp; &nbsp;<txp:posted />, <txp:rsx_time_of_day /></p>
19. <p><txp:body /><txp:if_custom_field name="referrer"> This link was found via <a href="<txp:custom_field name="referrer url" />"><txp:custom_field name="referrer" /></a>.</txp:if_custom_field></p><p><span class="float_left"><txp:comments_invite /></span><span class="float_right"><txp:rsx_frontend_edit_link suffix=" | ">Edit</txp:rsx_frontend_edit_link><txp:permlink>Perma-Link</txp:permlink></span>&nbsp;</p>
20. </div>
21. </txp:if_individual_article>
22.  
23. Download this code: /static/code/3.txt

Also, and I forgot to email this out to some people who asked, you need to keep track of how many articles are being displayed on your page. The first article, if it is a link, should also start a link block, and the last article on a page, if it is a link, should always end a link block.

In the doArticles() function in publish.php, you will need to add an article number variable. So after the if ($rs) { and before the foreach($rs as $a) { add the line $article_num = 0;.

In the for loop, after the $author has been set, add this line: $out['article_num'] = ++$article_num;.

This is the code I use to actually post to both del.icio.us and textpattern. I use it in conjunction with a HTML page that supplies the required POST variables. You can see what values need to be supplied via a POST request in the source code below.

1. <?php
2. include 'config.php';
3. define("txpath",$txpcfg['txpath']);
4.  
5. if (!file_exists('config.php')) die("There doesn't seem to be a config.php file. You must install Textpattern before you import any entries.");
6.  
7. include txpath.'/lib/admin_config.php';
8. include txpath.'/lib/txplib_db.php';
9. include txpath.'/lib/classTextile.php';
10. include txpath.'/publish/taghandlers.php';
11. include 'HttpClient.class.php';
12.  
13. $textile = new Textile;
14.  
15. // Set these values
16. $author = 'user';
17. $insert_with_status = 4;
18. $insert_into_section = 'linklog';
19. $override_form = 'Link-Post';
20. $annotate = 1;
21. $annotate_invite = '#';
22. $textile_body = 0;
23. $textile_excerpt = 0;
24.  
25. // Post variables
26. $title = $_POST['description'];
27. $body = $_POST['extended'];
28. $body_html = $_POST['extended'];
29. $keywords = $_POST['tags'];
30. $link_url = $_POST['url'];
31. $via = $_POST['via'];
32. $via_url = $_POST['via_url'];
33.  
34. // set URL title
35. $url_title = stripSpace($title);
36.  
37. // The Rest
38. $posted_date = date('YmdHis');
39.  
40. // check the url title is set
41. if (empty($url_title)) {
42. echo 'Error setting URL title from '.$title;
43. return;
44. }
45.  
46. // check if we need to do anything. we won't bother making a link with
47. // no title or url.
48. if (empty($link_url) || empty($title)) return;
49.  
50. // post to textpattern
51. $query = "INSERT INTO ".PFX."textpattern (
52. Posted, AuthorID, LastMod, LastModID, Title, Body, Body_html,
53. Annotate, AnnotateInvite, Status, textile_body, textile_excerpt, Section,
54. Keywords, url_title, override_form, custom_1, custom_2, custom_3)
55. VALUES
56. ('$posted_date','$author', '$posted_date', '$author', '$title', '$body', '$body_html',
57. '$annotate', '$annotate_invite', '$insert_with_status', $textile_body, $textile_excerpt,
58. '$insert_into_section', '$keywords', '$url_title', '$override_form','$link_url', '$via', '$via_url')";
59.  
60. echo '<!--'.$query.'-->';
61. safe_query($query);
62.  
63. // post to del.icio.us
64. $posted_date = date('Y-m-dTH:i:sZ');
65.  
66. $client = new HttpClient('del.icio.us');
67. $client->setAuthorization('user', 'password');
68. $sucsess = $client->get('http://del.icio.us/api/posts/add',
69. array('url' => $link_url,
70. 'description' => $title,
71. 'extended' => $body,
72. 'tags' => $keywords,
73. 'dt' => $posted_date));
74.  
75. ?>
76.  
77. Download this code: /static/code/4.txt

Web Design | Code