Tweet Scan Blog

Focus on Email Alerts and Backup, RSS Disabled

2010-01-10T16:35:00.000-08:00

Hello all,

A couple weeks ago, we disabled RSS feeds of our search results. Unfortuntely we don't forsee enabling them again, if you have a business need for specific RSS feeds, we may be able to re-enable them on a case by case basis. Otherwise, we ask that you remove any tweetscan.com/rss.php references from your web applications, Yahoo! Pipes, and other feed readers.

That being said, our web-based search, historical search, and Twitter account backup features should all be running quite a bit faster. In addition to those functions, we are also offering data export for specific keywords, phrases, and timeframes. If you have a question about any of the above, please email datasupport@NOSPAMtweetscan.com.

Thanks and Happy Scanning!

Twitter Backup now uses OAuth

2009-11-26T20:59:00.000-08:00

Our Twitter backup service has been widely used since we introduced our last update. That version combines your latest data from Twitter with what Tweet Scan has collected to make for the most complete Twitter backup available. Now, we are proud to release a new version that uses OAuth to connect to Twitter.

Now you can authorize Tweet Scan to access your account with read-only permissions which you can revoke at any time without having to play games with your passwords. If you haven't tried out our backup service, check it out at https://www.tweetscan.com/data.php. It's free if you tweet the tag #dlymydata or only $1.99 if you prefer not to tweet for us.

Happy Scanning!

We're back from a hard drive crash.

2009-06-23T14:23:00.000-07:00

On Friday, Tweet Scan's main database server crashed. I've spent the weekend rebuilding it and getting the backups restored. Unfortunately those backups were missing some data so all tweets from Feb 1st, 2009 to March 29th, 2009 have disappeared into the ether. That's about 2 months or 10% of the database.

The good news is, the service is now up and running 100%. Daily and weekly email alerts are back and operations are being improved so this doesn't happen again. Thanks for your patience and happy scanning!

Historical Search of 220 Million Twitter Messages

2009-06-10T13:33:00.000-07:00

Today, I humbly announce Tweet Scan Historical Search. Travel back as far as November 1st, 2007 in our 222 million message database. Subscribe

Here's what it can do for you:

Marketing departments, entrepreneurs, web developers, and bloggers can review their past successes of launching new products, features, and websites.

Researchers can study public response to business, political, social, and natural events.

Speakers, presenters, and event planners can review the chatter during their talk or conference and let it inform their next one.

The rest of us can use it to remember the good 'ole days or to find that long lost tweet that held your personal meaning of life. This library is a largest available and access is a mere $20/year. That's like a latte every other month or a small fraction of your phone bill!

Our database consists of 220,660,667 messages from Twitter and another 1,627,559 from identi.ca. It's still growing as you read this. You can search by keyword, by time down to the second, by microblogging service or by user. Access is available only to subscribers so register, subscribe, and you'll see a link on your control panel for Historical Search. You may want to bookmark this link right now: Historical Twitter Search. You'll also receive the other benefits of a Tweet Scan subscription including email alerts, personal archive download, and more features as we add them.

Thanks and Happy Scanning!

-David Sterry ( http://twitter.com/weex )

The Business Model

2009-03-22T09:16:00.000-07:00

Tweet Scan is entering a new phase of becoming a sustainable business.

How do we plan to do that? First, to provide better service we will charge for an account. That will be followed by increased development in which we plan to cover more microblog networks in email alerts, offer more saved searches, increase search speed, and help you with the kinds of analysis you want to do. Any improvements we make will be available as part of your paid membership.

The cost is $15/year if you join or upgrade before April 23rd (with the early-adopter discount). On April 23rd, the fee will rise to a standard $20/year. If you do not upgrade your account by April 23rd, email alerts and access to the site will be suspended.

If you get already our daily email alerts, it's a great deal. If not, we urge you to join or upgrade today. Many of your colleagues are using Tweet Scan to stay aware of their brands, websites, and interests. You and they will benefit more as the service evolves.

Visit http://tweetscan.com/main.php to join or upgrade.

User Search

2008-11-04T12:42:00.000-08:00

Looking for people to follow on Twitter? Not sure you want to follow based on a keyword used in an off-hand comment? Then try our new User Search! This searches the Twitter profiles we've indexed for their location, bio, and the beginning of the username. Give it a try and find some new folks to follow.

Once your done loading up on people to follow, join our site and get your free daily search emails started!

Happy Scanning.

Search Extended beyond Twitter and identi.ca

2008-10-19T16:34:00.000-07:00

A while back, we added identi.ca messages or "dents" to our search index. It was and still is accessible at a second search page(tweetscan.com/identica). Since then, we've received requests that it be integrated to search both Twitter and identi.ca in one search result.

We've taken this idea a step further by integrating not just these two major Microblogging sites but also eight other Laconica based sites. These sites are in no particular order: bleeper.de, army.twit.tv, lareta.net, linuxoutlaws.net, sufficetosay.frontrowcrew.com, micro.ciarang.com, fun.ohhh.no, and pawwer.com. None of these gets the volume approaching Twitter at the moment but we think it's a great thing for the microblogging space to have them somehow unified in search.

For the moment, RSS doesn't do all these sites but you can do Twitter and identi.ca by visiting the API page to read about that. Also, if you know of another major site we should be indexing, feel free to comment or message @tweetscan or @weex.

Thanks,

David Sterry
http://tweetscan.com

2008 Debate #1 Twitter Statistics

2008-09-26T21:02:00.000-07:00

The first Presidential debate of the 2008 Election started about 3 hours ago and since that time, we've captured 142,002 messages by 50,792 users from Twitter's public message stream. We present here the number of times some words of interest were mentioned during that time. For reference, the word "debate" was counted 12,460 times.

If you have any questions, feel free to add a comment. To follow future debates in real-time, bookmark this link.

Download Your Twitter Message Archive

2008-09-12T08:34:00.000-07:00

We are very excited about a new product we released. You can now download your Twitter message archive consisting of any messages we've captured from our starting date of December 1st, 2007 to today. Just go to https://www.tweetscan.com/data.php to order. You can choose to download your messages alone or to include your replies as well. Only public messages we've previously captured are included in the database, though we believe it is the most comprehensive outside of Twitter itself.

We appreciate any suggestions you have to improve the service. Please email them to datasupport@tweetscan.com. If you are the first to come up with a suggestion and we choose to implement it, we will provide you with a complementary message archive with replies.

Thanks,

David Sterry
TweetScan.com

Ubiquity for Tweet Scan

2008-08-29T16:51:00.000-07:00

Have you heard about Ubiquity? Mozilla's new plugin for Firefox adds a command line interface to their great web browser. Since Tweet Scan provides some web services, I thought I'd try my hand at creating a command to let people execute a search. To try it out, visit http://tweetscan.com/api.php and subscribe.

After installation, restart Firefox, invoke ubiquity and start typing 'tweetscan'('twe' is all you really need to type in a default install). Now, a Firefox restart isn't supposed to be required so if anyone knows how I can remove that step, I'm all ears.

Creating the command wasn't hard. All I did was take the 'echo' example from the author tutorial, use CmdUtils.getDocumentInsecure to grab the current tab, and set the document.location to the search url after escaping the search phrase. You can take a look at the code when you subscribe or by going to http://tweetscan.com/tweetscan.js.

This is the simplest command I could think of for Tweet Scan but I'll be looking for a way to extend this to make Tweet Scan useful for email and other creative contexts.

Happy Scanning,

David Sterry

identi.ca support

2008-07-22T23:57:00.001-07:00

Tweet Scan has recently begun tracking dents on the identi.ca network. If you've not heard of identi.ca, on the face of it it's a Twitter clone. But in actuality it is the main installation of the first major piece of Affero GPL'd software to hit the web: laconi.ca.

If you'd like to search messages on identi.ca, proceed to http://tweetscan.com/identica

At this moment, rss feeds, email alerts, and other parts of Tweet Scan are still only pointed at Twitter but we plan to grow our support for identi.ca as the platform matures.

API: JSON, RSS and Badges

2008-05-02T09:03:00.000-07:00

What does API stand for? Does it matter? Truth is, a web service just isn't a web service without an API. Informally, anybody could use the RSS feeds we've been providing as an API but well there are easier ways. Especially when it comes to creating other web applications that process and display real-time information.

So we've created a JSON implementation. If you are not familiar with it, JSON is a format that is ready to read by Javascript. In our case, we provide that data wrapped in a call to a function called ts_parse. If you define that function to handle the data, that's a start. You can look at http://tweetscan.com/badge.js for a reference implementation. Which leads us to the badges....

The first use of the JSON and what was used for testing is also available at http://tweetscan.com/api.php in the form of a badge. You can grab a couple snippets of code, put in your search string, and wham! you'll have topical live updating tweets on your site.

Of course, we'd love to see what else can be done with this data. The possibilities are nearly endless. So if you've got a developer's itch to scratch, give it a shot and let us know if you need anything more in the API.

Happy API Scanning!

Video of Tweet Scan

2008-04-29T00:20:00.000-07:00

Just saw this cool video today from Robert Scoble's feed on Qik. He talks about how he uses Twitter and Tweet Scan(2:22) so look, listen and learn. Robert mentions searching: Microsoft, Scobleizer, Winer, and retweet.

Search by date

2008-04-27T10:35:00.000-07:00

"History, history is laughing at us, plotting its discovery, victory, victory, blame it on the victory." - Bad Religion

Today marks the first day where all previously collected tweets are available online. You can search back in time by clicking on the "just now" field and choosing a date from the calendar. You can also type the date in YYYY-MM-DD format if you prefer the keyboard which we often do.

Why is this useful? You might be doing a report for a client and want to judge consumer sentiment before and after launch of the new site. Or perhaps you remember a friend said something cool but it was so far back that paging through your archives is impractical. Maybe you're just a historian trying to help us all avoid repeating our mistakes.

This is a new feature and there may be bugs so please do tweet about them including the word "tweetscan" so we can log them, submit to code review, and finally to our implementation group to be rolled in with the next Tweet Scan service pack. ;)

Happy Scanning!

Redesign and Popular Searches

2008-04-15T17:32:00.000-07:00

If you've visited the site in the last couple hours or if you helped test it yesterday, you are now seeing a redesign of the site. The previous interface was a hack, too spacious, and even we were getting tired of that green. Tweet Scan now takes up less space on the screen and uses strict(er) css for a standard layout no matter which browser you prefer.

Additionally, we wanted to put a new thingy(that's an accepted word now right?) on the page. Google has their Zeitgeist which is cool but having something that updates once a month or even once a week is glacial by Twitter standards. Our Popular Searches div updates once an hour and reflects some of the most popular searches of the last 24 hours.

Happy scanning!

The OR operator

2008-04-01T23:25:00.000-07:00

In the quest to make our humble search engine the best it can be, we've implemented the OR operator in our search. So if you wanted to search for cats or dogs you can enter that as 'cats OR dogs'. If you want to do an OR between two exact phrases or an exact phrase and a single word, that should work too. For example 'cats OR "big dogs"' will work as you'd expect. Note that the OR must be capitalized for it to work.

This updated functionality should work in RSS, on the site, and in Twhirl. If you have any comments or suggestions, let us know at @tweetscan OR right here in a comment.

Happy scanning!

RSS Improvements and Wiki

2008-03-17T21:45:00.000-07:00

With Terraminds down over the past few days, many people have been looking for an alternative for their Twitter RSS search feeds. One problem with switching to Tweet Scan for RSS was that our feed didn't include the pubDate field in the feed and at least one web application(TwitterFeed) rejected the feed. This has been fixed.

Also, there are a few advanced features of Tweet Scan like boolean 'and' and 'not' features that people ask for but that are already in there. So we decided it was time to put up a wiki for the project. You can read it(and it is a quick read) at tweetscan.com/wiki. We're particularly interested to hear if you've found some unique uses for Tweet Scan that we didn't think of.

Happy scanning!

Tweet Scan Core

2008-03-10T16:00:00.000-07:00

You follow me, I follow you. On Twitter follower reciprocity is a custom, and at times a rule. It's nice to have lots of followers but if Twitter is a tool to bring us together, this moré can work against us.

It can turn our home page into a virtual firehose of disconnected thought and status. That's why we've developed Tweet Scan Core. It's a simple way to save users that you'd *really* like to follow and see each of their latest tweets from one page. It's your core network status in real-time.

To get it, just sign up at Tweetscan.com and build a comma-separated list of twitter users like this: "tweetscan,weex,jowyang,monkchips". Save your settings and visit http://tweetscan.com/core.php as often as you'd like.

If you have any suggestions, message @tweetscan or leave a comment.

Happy scanning!

TweetScan for iPhone

2008-02-16T00:10:00.001-08:00

Thanks for checking out the blog. The latest news is that we got an iPhone for "development" purposes and we just had to do something cool with it. So we present the iPhone version of TweetScan. If you have an iPhone, you don't have to do anything special to use. We just check the user agent for "iPhone" so if you want to see it, you know what to do.

To create the iPhone version, we:
1. Removed the install search link.
2. Moved the link bar below the logo.
3. Reduced the font size.
4. Reduced the page width.

This version may be useful for other mobile/small screen devices so perhaps the next step we'll take will be to look for more user agent strings. If you have a particular device you'd like to see included, message @tweetscan or leave a comment somewhere on the interweb.

Happy scanning!

Search Twitter Posts by User

2008-01-02T00:58:00.000-08:00

Tweet Scan is useful for finding replies that Twitter doesn't or for easily following topics and conversations you care about. There is another feature that people keep requesting, and it's a pleasure to fulfill that request. People(you) want to be able to search tweets by themselves or another user. It's like searching your own email, a must-have feature.

Testing it has been a lot of fun. For example, I wanted to revisit the link codinghorror posted in this post but there was really no good way to do it before now. With a simple search for "mouse" by "codinghorror" it was quite easy to find.

That's why I present to you the ability to search posts by user. Try it out and send a shoutout to @weex or just mention tweetscan if you like it. Happy scanning.

Happy New Year

2007-12-31T20:49:00.000-08:00

Since Twitter's nearly defined by its instant timeliness, I figured I'd post a couple good searches for Now.

First there are the New Years resolutions which are already a popular tweet subject. And one other that's a fav of mine is due to their live webcast starting in 9 minutes as of this time of writing: Radiohead

Here's to 2008. May it be even more awesome than 2007 was.

Happy Holidays: Apps for Tweet Scan

2007-12-23T22:25:00.000-08:00

Happy Holidays. It's been a great year for Twitter and the ecosystem that has grown up around it. From Snitter to Twurt to Tweet Scan, Twitter has proven an exciting service and a great bit of social glue for its members.

This post, I wanted to just mention a few uses of Tweet Scan that we couldn't have anticipated but that we think are super sweet. Don't we all want to be loved?

1. Replies - sometimes Twitter doesn't show the replies to your post if other @users were also replied to. Searching for your username, brings 'em all in.

2. Twurt - from the site: A mashup of the words “tweet” and “blurt”...have you twurted?

3. Overheard.it - looking for a little company when you hear something outrageous? Just preface your tweet with oh: or overheard and the voila, all of twitter just overheard it too.

We look forward to supporting these and many more apps in the new year. Happy scanning.

RSS now available for any search

2007-12-04T00:34:00.000-08:00

While quite a few of you have tried out the Click button, it's still evident that RSS is a must-have. So we've implemented it.

Subscribe to any search you can think of by clicking on the RSS link on the upper right of the search results. RSS is not our forte so if something's broken about it or you think it could be a little smoother in some way, please let us know.

Click through your saved searches

2007-12-03T07:46:00.000-08:00

One great motivator when building any kind of tool is "eating your own dog food". In our case that food tastes pretty good, could use a little salt, but is quite nutritious. My point? With Twitter and Tweet Scan we've all got a great user feedback loop.

One of the first feedback items for us is to implement RSS. While it's definitely doable to serve RSS, it's not #1 on our list of exciting things to do. RSS is popular and widely used but we're trying something else, something more...how do you say? fun.

We've created a link that will simply run each of your saved searches in sequence. This was inspired by Google Reader's Next button(the most fun we ever had with RSS btw). So next time you find yourself at your settings page, copy that "Click" link to your bookmarks. Give it a click and vote your opinion in the form of this tweet.

Tweet Scan tames World's Largest Water Cooler

2007-11-20T23:54:00.000-08:00

If you like to read your news on the Interwebs(and who doesn't?) then you might find Tweet Scan a nice complement to the usual browse. Just type in a timely topic and you're likely to find several discussions and blogs about that story. Twitter is an enormous water cooler and Tweet Scan can help you track the gossip you care about.

For example, read about Kindle or the new Firefox beta.

Happy scanning!

Added Open Search plugin

2007-11-18T22:09:00.001-08:00

If you offer any kind of search, you've got to have a plugin so people can search your site without having to actually go there. That's why we've introduced an Open Search feature to Tweet Scan. For a while, you'll see a link on the search page that lets you add Tweet Scan to your browser. Happy scanning!

"Tweet Scan's fab"

2007-11-09T17:42:00.000-08:00

So say the folks over at Real Fresh TV.

Since we've just gotten started it's obviously an honor to be mentioned positively so early in our evolution. It's our goal to be the best Twitter Search Engine out there and we look forward to user feedback that will help us make an impact on the Social Networking and Twitter scene.

If you'd like to try it out, go to tweetscan.com. There you can search public Twitter posts in real time and sign up for email alerts for the searches that matter to you.

2007-11-08T17:41:00.000-08:00

Tweet Scan is now capturing all public tweets.

For the past month or so, I've been building and testing Tweet Scan and capturing enough tweets to make thing interesting. A search tool is supposed to find what people want and Tweet Scan was missing probably 95% of tweets.

So I've just updated the capture engine to get as close to 100% of public tweets as possible. If you think it's missing a recent tweet or if there's a feature you'd like to see, send a tweet to @tweetscan or comment here.

If you like what you see when you search, you can sign up here for any search to be run automatically and emailed to you when you choose.

Search Twitter with Tweet Scan

2007-10-25T17:39:00.000-07:00

Today, I'm releasing Tweet Scan for your enjoyment. Tweet Scan emails you every day or week with the Twitter messages you choose.

Twitter has given me something fun to do with my Treo's web browser and for that I thank them. My special way of thanking goes beyond just putting my hands together gratefully and looking toward the sky. I created Tweet Scan. It allows you to search recent Twitter messages for words and phrases of interest to you. It's like a mini-Google for Twitter.

You can search in near real-time, the messages appearing Twitter's public timeline xml file. At this time I figure about 1/5th of posts are recorded there so don't be surprised if nost of your posts don't appear in the search results. If enough people sign up for Tweet Scan, I'll flip a switch so almost every public message is searchable.

Beyond a simple search engine, you can get daily emails of tweets that mention your interests. When you're choosing your search terms, it's easy to preview the results. Once you're setup, you'll get a daily email including tweets that match. They're really quite fun to read through if you choose good keywords. If you prefer more infrequent emails, you can choose the weekly option.

It's free and fun so go ahead and sign up. Then you'll have some fun email to read the next time you're waiting in line or sipping away at your local coffee shop.

I'd love to hear your feedback on Tweet Scan. Just shoot me an email at tweetscan at davidsterry.com