As I suspected. The problem is that we can't easily cache all pages for logged in members all the time, because there are so many pages and so many members. 

We're looking at ways to minimize the wait time, and until we do the same rule applies: only log in if you want to add content. We are able to cache almost all pages for visitors who are not logged in.

Marshall

You shouldn't have to cache pages for logged in members. If this is the only way you can get performance, there is something else wrong. I have used other Plone sites in the past that are really fast. Again I suspect it is the database server, database index, or other related database item. I have worked on some large scale websites that handled millions of unqiues a month. When we had performance problems, 98% of the time it had to do with the database stored procedures, indexes, or database architecture. Very rarely it had to do with not enough hardware.

This has been a problem for a long time. I suspect alot of people (myself included) don't participate in the community as they used to because it is too painful to wait 10 secs everytime you want to look at a page. Or log out and log in all the time.

I would spend a couple thousand to get a Plone expert / database expert to track down the problem. You will get it back in increased page views / ad revenue over time and increase the participation of the community. I would love to participate more, but it takes too long.

Jason,

I've had the database checked by experts (which is object oriented, not SQL), and they can't find any obvious problems, except for its size. There are ways that we can reduce its size (putting images on the filesystem), and we'll explore those over the next few weeks.

One thing we could do right away, is to open up the forums to allow you to add comments without logging in. There are some trade-offs, but the upside is that speed is much faster if you are not logged in. We'll chart this course and implement if the downsides are solvable (i.e., preventing spam).

Marshall