Efficiently pulling data out - Unfuddle Community Forums

Apr 3, 2008 11:08pm

Todd Geist 14 post(s)

Hello,

I am running a Cron Job that imports all my unfuddle data into another database. Currently this is very inefficient, because I am pulling all of the project data including tickets, time entries, etc, even if this data has not changed since the last time I ran it. I would like to only pull data for objects that have changed. That way i could significantly reduce my hits on the server.

I have thought about using the Activity report for the account, but I don’t think it includes all the objects.???

has anyone come up with another way to pull data more efficiently???

Thanks

Todd

Apr 5, 2008 2:08am

Todd Geist 14 post(s)

Well I went back and looked at the account activity report. Parsing that is far less damaging to the server.

Apr 5, 2008 5:04am

Joshua Frappier Administrator 241 post(s)

Todd,

I would say that using the activity on your account or project is a very appropriate way to determine the activity that has happened over a specified period of time.

Good luck with your integration!

Josh

Apr 5, 2008 5:33pm

Todd Geist 14 post(s)

activity is “OK”, but it misses a lot…. For example updates to time-entries are not there.

What would be ideal is an api call that returned all of the new or updated objects between two time stamps. I could hit the server just one time and then go do my updates.

Something like.

ap1/v1/account/changes.xml

You are always going to have people who are going to want very current data in another system, and providing something like that will give those people a way to do it with out hammering your server to hard.

thanks

Apr 8, 2008 1:53am

samreid 17 post(s)

I’ve had to address the same issue with a cron job. I’ve been using the activity api to draw the last N items, manually deciding which are new, and writing to a log file to keep track of issues that have been handled. It would have been more natural and convenient for me to access these items by specifying a time range.

Apr 3, 2008 11:08pm Todd Geist 14 post(s)	Hello, I am running a Cron Job that imports all my unfuddle data into another database. Currently this is very inefficient, because I am pulling all of the project data including tickets, time entries, etc, even if this data has not changed since the last time I ran it. I would like to only pull data for objects that have changed. That way i could significantly reduce my hits on the server. I have thought about using the Activity report for the account, but I don’t think it includes all the objects.??? has anyone come up with another way to pull data more efficiently??? Thanks Todd

Apr 5, 2008 2:08am Todd Geist 14 post(s)	Well I went back and looked at the account activity report. Parsing that is far less damaging to the server.

Apr 5, 2008 5:04am Joshua Frappier Administrator 241 post(s)	Todd, I would say that using the activity on your account or project is a very appropriate way to determine the activity that has happened over a specified period of time. Good luck with your integration! Josh

Apr 5, 2008 5:33pm Todd Geist 14 post(s)	activity is “OK”, but it misses a lot…. For example updates to time-entries are not there. What would be ideal is an api call that returned all of the new or updated objects between two time stamps. I could hit the server just one time and then go do my updates. Something like. ap1/v1/account/changes.xml You are always going to have people who are going to want very current data in another system, and providing something like that will give those people a way to do it with out hammering your server to hard. thanks

Apr 8, 2008 1:53am samreid 17 post(s)	I’ve had to address the same issue with a cron job. I’ve been using the activity api to draw the last N items, manually deciding which are new, and writing to a log file to keep track of issues that have been handled. It would have been more natural and convenient for me to access these items by specifying a time range.