Just came across this article from a while back. I've been thinking about some of these issues for a while and was wondering if there need to be policies to work around them? Usability guidelines like Olli's making are good, but some of the problems listed here are systemic and guidelines won't necessarily help e.g. the tendency for too many preferences to be added.
The article reminded me of a point Tim raised in his thoughtful blog post last week about the views of geeks and the complexity of e.g. the update quiz interface. At the time, I agreed that the fact that teachers were keen on keeping all of the preferences showed that they were valuable. Now, I'm not so sure and I think it may be an artefact of the open source process which leads to lots of minor settings being added, which overwhelms the UI.
I'm particularly aware that discussing whether to change things mostly happens on these forums and that we are all, by definition, the power users and therefore not the best people to be judging things. I think what's needed is some formalised way of testing new interfaces with novice users. I'd be happy to get the opinions of my colleagues and students (the ones who are a bit technophobic) if there was some way of feeding that information back in a standardised and useful way.
"Why open source software usability tends to suck" - how do we work around this?
by Matt Gibson -
Number of replies: 35
In reply to Matt Gibson
Re: "Why open source software usability tends to suck" - how do we work around this?
by Marc Grober -
Not to in any way detract from your point, bug the downside in some respects of the forums is that the sane points are raised every x months.
I recall a similar discussion regarding the gradebook where if was suggested that as part of the QA process a variety of users (teachers, admins, students) at various levels of sophistication be recorded working with development projects with comments appropriately attached....
There has been I think some greater focus on providing user testbeds, but as I think you are noting, tester selection is far from random or broadbased.....
On the otherhand, there have been quite a few comments about making the GUI extensible and adjustable from inside moodle so that the quiz ui, for example, can be easily customized with the features sought by Tom, Dick, or Harry. I had a similar challenge while moving green screen apps to the web. Under RPG any screen change required an IT work order... But users, provided with some simple tools and limitations could fiddle the web GUI to their hearts content...
It was based on this kind of idea that I argued for "templates", i.e. the ability to afford presets that would take advantage of admin menus (much in the way tinyMCE is configured)
admin menus, presets, enhanced QA, etc are not new ideas; neither is the tension between the coder who never seems to hear the user and the user who can't seem to be able to make himself understood to the coder.....
I recall a similar discussion regarding the gradebook where if was suggested that as part of the QA process a variety of users (teachers, admins, students) at various levels of sophistication be recorded working with development projects with comments appropriately attached....
There has been I think some greater focus on providing user testbeds, but as I think you are noting, tester selection is far from random or broadbased.....
On the otherhand, there have been quite a few comments about making the GUI extensible and adjustable from inside moodle so that the quiz ui, for example, can be easily customized with the features sought by Tom, Dick, or Harry. I had a similar challenge while moving green screen apps to the web. Under RPG any screen change required an IT work order... But users, provided with some simple tools and limitations could fiddle the web GUI to their hearts content...
It was based on this kind of idea that I argued for "templates", i.e. the ability to afford presets that would take advantage of admin menus (much in the way tinyMCE is configured)
admin menus, presets, enhanced QA, etc are not new ideas; neither is the tension between the coder who never seems to hear the user and the user who can't seem to be able to make himself understood to the coder.....
In reply to Matt Gibson
Re: "Why open source software usability tends to suck" - how do we work around this?
by sam marshall -
At the Open University we do periodic user testing (by employing some of our real students to test). However, we generally concentrate on our own new features rather than Moodle core.
By the nature of user testing, it is not at all a reliable method for drawing firm conclusions 'is this usable' or 'does this suck'. This is true of virtually all user testing. The reason is that user testing takes an extremely small sample of people, who are self-selected and may not be entirely representative (i.e. we don't offer that much pay, so you have to be either really in need of money, or really motivated to help us improve) and puts them in an unrealistic situation, etc. However, you can learn valuable individual points from the individual users involved. Also, ours should be not-that-unrepresentative because our students are generally adults from a wide age range etc. (As compared to studies using students in a 'traditional' university where they're all young adults.)
When I see videos of user testing I am always shocked. Here are two recent fun examples:
1. Student who couldn't find the 'save' button on a form because it's off the bottom of the screen (yes, he knew how to use scrollbars... he just didn't think to).
As a lesson learnt from this, we are placing a solid background colour on all forms in our theme (instead of the current thin border). My hope is that, because this clear chunk of background colour extends right to the bottom of the visible display, it will make clear that there is more form afterward. I don't know if this will actually work or not
Hope so...
2. Student who couldn't figure out how to create a page in our wiki. He clicked the help button next to 'Help with creating pages', and appeared to be reading the popup help, but did not get as far as the heading midway down the help screen 'Steps to create a wiki page' or the 1, 2, 3, 4, 5 numbered list below it.
The lesson learnt from this basically that not only do people usually not read online help, but even when they think they're reading online help, they might not be. Basically we need to make sure that all necessary information is available on the main screen, while keeping it extremely short (or people won't read that either).
In this case we are adding a 'Create new wiki page' button at the bottom of the wiki screen.
Obviously our students are not at all stupid (they are studying at university level). So the third thing to learn, from these examples and others, is that end users, without being stupid, are incredibly bad at using computer software. I mean that in the literal sense of incredible - as in, they are so bad that we as developers and expert users actually can't believe it.
And it's not just some people who are bad at using computer software - as I said, these usability tests involve very small samples, but I don't remember seeing a usability test report with a single person who didn't have huge trouble using at least one relatively straightforward part of the software (e.g. scrolling, or reading help).
One other thing I wanted to say about user testing is that any suggestions that test users make directly will almost certainly be unhelpful. They don't know how to use the software - why should we expect them to design it for us? But that's not a problem - the valuable information they provide is basically what they do (and what they say about why they couldn't figure it out). From that point it's up to designers to come up with feasible ideas that might improve the situation.
And finally, I think user testing can be helpful but it's slow and expensive and should probably be the last resort. There are lots of cases where you can already see (without involving users) that something really sucks; there's no point paying people to tell you that. Fix it first, then do user testing... But in our case, user testing also provided a justification to schedule the development work to make these improvements.
--sam
PS As with everything I say, these are all my own opinions and not my employers'
By the nature of user testing, it is not at all a reliable method for drawing firm conclusions 'is this usable' or 'does this suck'. This is true of virtually all user testing. The reason is that user testing takes an extremely small sample of people, who are self-selected and may not be entirely representative (i.e. we don't offer that much pay, so you have to be either really in need of money, or really motivated to help us improve) and puts them in an unrealistic situation, etc. However, you can learn valuable individual points from the individual users involved. Also, ours should be not-that-unrepresentative because our students are generally adults from a wide age range etc. (As compared to studies using students in a 'traditional' university where they're all young adults.)
When I see videos of user testing I am always shocked. Here are two recent fun examples:
1. Student who couldn't find the 'save' button on a form because it's off the bottom of the screen (yes, he knew how to use scrollbars... he just didn't think to).
As a lesson learnt from this, we are placing a solid background colour on all forms in our theme (instead of the current thin border). My hope is that, because this clear chunk of background colour extends right to the bottom of the visible display, it will make clear that there is more form afterward. I don't know if this will actually work or not
2. Student who couldn't figure out how to create a page in our wiki. He clicked the help button next to 'Help with creating pages', and appeared to be reading the popup help, but did not get as far as the heading midway down the help screen 'Steps to create a wiki page' or the 1, 2, 3, 4, 5 numbered list below it.
The lesson learnt from this basically that not only do people usually not read online help, but even when they think they're reading online help, they might not be. Basically we need to make sure that all necessary information is available on the main screen, while keeping it extremely short (or people won't read that either).
In this case we are adding a 'Create new wiki page' button at the bottom of the wiki screen.
Obviously our students are not at all stupid (they are studying at university level). So the third thing to learn, from these examples and others, is that end users, without being stupid, are incredibly bad at using computer software. I mean that in the literal sense of incredible - as in, they are so bad that we as developers and expert users actually can't believe it.
And it's not just some people who are bad at using computer software - as I said, these usability tests involve very small samples, but I don't remember seeing a usability test report with a single person who didn't have huge trouble using at least one relatively straightforward part of the software (e.g. scrolling, or reading help).
One other thing I wanted to say about user testing is that any suggestions that test users make directly will almost certainly be unhelpful. They don't know how to use the software - why should we expect them to design it for us? But that's not a problem - the valuable information they provide is basically what they do (and what they say about why they couldn't figure it out). From that point it's up to designers to come up with feasible ideas that might improve the situation.
And finally, I think user testing can be helpful but it's slow and expensive and should probably be the last resort. There are lots of cases where you can already see (without involving users) that something really sucks; there's no point paying people to tell you that. Fix it first, then do user testing... But in our case, user testing also provided a justification to schedule the development work to make these improvements.
--sam
PS As with everything I say, these are all my own opinions and not my employers'
In reply to sam marshall
Re: "Why open source software usability tends to suck" - how do we work around this?
by Peter Seaman -
The problem with most usability testing, in my experience, stems from the way the testing situates a user in a moment in time and doesn't account for increasing skill over time (as a user gains experience with an application). We tend to find a representative user - especially one who is not "tainted" by prior experience - and we expose her to the application for the first time, and then she struggles with it (of course).
I once had the good fortune to participate in a software-development project in which I served on a user-testing focus group. The group met initially to tell the developers what we wanted the application to do. The very skillful project manager said at the time, "Don't be surprised if your specs change over time. You think you know what you want, but you don't actually know what you want, and this is normal." He was absolutely right! The focus group met every two weeks and reviewed the development of the software, and we learned over time how it worked and how we wanted it to work (both are moving targets, with the danger of "mission creep," as others have noted). If Moodle uses anything like the latter process, I would think it's much more useful than isolated usability testing, which tends to provide rather obvious insights that have less application for the actual user, whose skills evolve over time. Thanks. - Peter
In reply to Peter Seaman
Re: "Why open source software usability tends to suck" - how do we work around this?
by dawn alderson -
Peter, thanks for this-interesting
re the project....user testing with software devs....so were you seeing it from teacher or learner- or both stances?
Or admin? I am curious to know.
thanks,
Dawn
In reply to dawn alderson
Re: "Why open source software usability tends to suck" - how do we work around this?
by Daniel Neis Araujo -
Hello, everybody
nice to have this forum back.
Good to read things from a time were 2.0 was still on horizon =)
In fact, 2.0 was a major improvement on usability and everything else.
After that, droping all old themes and the adoption of Bootstrap Framework for the new Clean and More standard themes on 2.7 was another important milestone for Moodle.
The next big improvement we will have will be on 2.9 (maybe 3.0), mostly for programmers and designers, as Moodle is updating it's Javascript "libraries and framework" as discussed on https://moodle.org/mod/forum/discuss.php?d=268190 and https://docs.moodle.org/dev/JS_Framework_Specification and https://tracker.moodle.org/browse/MDL-47036
Of course it is more on technical/practical side than the more theoretic/philosofical/methodological talk we are having in old posts, but I think an update would fit here =)
Also, as noted by Matt Gibson back on 2009:
"A specifically usability based bugathon style event that would first identify the dozens of little annoyances, rank them in order of annoyingess and then iron them out."
We have a lot of issues on tracker, maby alreasy closed but many still open
so it may be a start point:
| Status | Issues | Percentage | ||
|---|---|---|---|---|
| Open | 256 |
|
||
| Reopened | 10 |
|
||
| Development in progress | 5 |
|
||
| Waiting for peer review | 3 |
|
||
| Closed | 798 |
|
Now we can begin starting forum discussions to all those interestings features and find the better way to approach them =)
Kind regards,
Daniel
In reply to Daniel Neis Araujo
Re: "Why open source software usability tends to suck" - how do we work around this?
by Colin Fraser -
Daniel, you already have an audience, and testers, and a proven method of communication, that is the Moodle Community, and these forums. I have to ask what are the most frequently asked questions for assistance? What are peoples' peeves? For example, when someone like Derek C starts talking about the awkward handling of video, why isn't there a more positive response? If Derek is speaking out, then there might be an issue. These, I suggest are only indicators, they are not, of themselves, usability issues. Many issues are resolved easily, simply, but why have they caused a problem in the first place?
One of my concerns right now is that the release schedule is so tight that the innovation it is driving might be interfering with the necessary polishing that should be done with supplementary releases. e.g. instead of a v2.9, how about a v2.8.2, then a v2.8.3, then a v2,8.4 and a v2.8.5 and so on, maybe to a v2.8.9.20.
Sometimes it is necessary to stop and smell the roses...
In reply to Colin Fraser
Re: "Why open source software usability tends to suck" - how do we work around this?
by dawn alderson -
oh crumbs Colin!
re: One of my concerns right now is that the release schedule is so tight that the innovation it is driving might be interfering with the necessary polishing that should be done with supplementary releases. e.g. instead of a v2.9, how about a v2.8.2, then a v2.8.3, then a v2,8.4 and a v2.8.5 and so on, maybe to a v2.8.9.20.
doesn't this already happen? https://docs.moodle.org/dev/Moodle_versions
now Colin-that should be crystal clear to you...because it is to me OK
And it is linked to the roadmap too....so again clarity there for you....and don't ask me how it is linked to the roadmap.....each version that is...because you really ought to know that too
OK. While I am on the topic then......I think......yes I really do, that progress to V.3 might focus on a parallel process involving web. 3 integration........shhhh...it is a massive secret so dont tell anyone OK Colin. And, I think, well my grandmother used to say this.....it is good not to collect too much brass...because otherwise you just end up with too much to polish!
cheers,
D
In reply to Matt Gibson
Re: "Why open source software usability tends to suck" - how do we work around this?
by Olli Savolainen -
Thanks, Matt for opening this discussion.
I think your thought that bad design it is an artifact of open source, is interesting. I would be inclined to think so, too. But then, how do strong designs like Firefox or Gnome appear? The answer it seems, is that open source does not exclude great design. There simply has to be a strong vision about which user goals of which users we are to serve. On one hand, Moodle started from broad user goals - on the other, those needs are not explicitly stated anywhere in enough detail to drive design. In Cooper's terms, we are riding a tiger: users dictate what we do, since we lack a strong enough vision ourselves. But strangely, Moodle is somewhat successful. In Cooper's terms: the wonder is not that the bear dances well, but that it dances at all. Joel Spolsky also had his say about this.
Both the arts of requirements engineering and of user centered design promote the idea that we do not do great design by asking users what they need, but by really understanding their actual goals and being capable of translating those needs into designs. Users only know what they need when they see it done.
In practice this means that the two jobs need to be separate. Programmers used to do their own testing before it was discovered that it requires a different skill set and it is more productive to save the programmers' time for development and architectural design. similarly designing user interaction is whole another ball game, it makes sense to really stop and think about the division of work. This is in no way contradicts making open source software.
Then, when new feature requirements come up that we did not think of, we need to see if we understand that usage need already or if we need to research the context of the goals the feature requirement reflects. When we understand what the users are actually doind, we can
a) decide if the user groups really important to us really need this feature
b) decide how to integrate this feature in a way that is just prominent enough, relative to its importance - an appropriate balance such that it does not distract existing users.
This way, we don't end up just piling features on top of each other in single screens, making users tell the software whether they need those features or not. Since we know the goals of our users, we can make those decisions for the users and let them concentrate on what they really want to do.
When we don't have real data, we end up feeling that it is best that users decide for themselves, which is not what they paid for. Which, of course, is nothing, so we end up in a "collaborative" design process where the user makes design decisions for us, when using the product, degrading the user experience. The user interface tries to serve everybody equally, and with same kind of interaction for all kinds of users. One size fits all fits nobody well.
I am only just learning myself how to really study and model those needs in a way appropriate for open source.
I just recommended Cooper's The Inmates Are Running the Asylum to Tim last week, since it discusses the question, starting from the roots of the issue, profoundly, still being a relatively short and an entertaining read. I warmly recommend it to anyone taking part in this discussion. (I am not sure but the example above about division of work might have been from that book.)
Apple Human Interface Guidelines: Designing for the 80%
I think your thought that bad design it is an artifact of open source, is interesting. I would be inclined to think so, too. But then, how do strong designs like Firefox or Gnome appear? The answer it seems, is that open source does not exclude great design. There simply has to be a strong vision about which user goals of which users we are to serve. On one hand, Moodle started from broad user goals - on the other, those needs are not explicitly stated anywhere in enough detail to drive design. In Cooper's terms, we are riding a tiger: users dictate what we do, since we lack a strong enough vision ourselves. But strangely, Moodle is somewhat successful. In Cooper's terms: the wonder is not that the bear dances well, but that it dances at all. Joel Spolsky also had his say about this.
Both the arts of requirements engineering and of user centered design promote the idea that we do not do great design by asking users what they need, but by really understanding their actual goals and being capable of translating those needs into designs. Users only know what they need when they see it done.
In practice this means that the two jobs need to be separate. Programmers used to do their own testing before it was discovered that it requires a different skill set and it is more productive to save the programmers' time for development and architectural design. similarly designing user interaction is whole another ball game, it makes sense to really stop and think about the division of work. This is in no way contradicts making open source software.
Then, when new feature requirements come up that we did not think of, we need to see if we understand that usage need already or if we need to research the context of the goals the feature requirement reflects. When we understand what the users are actually doind, we can
a) decide if the user groups really important to us really need this feature
b) decide how to integrate this feature in a way that is just prominent enough, relative to its importance - an appropriate balance such that it does not distract existing users.
This way, we don't end up just piling features on top of each other in single screens, making users tell the software whether they need those features or not. Since we know the goals of our users, we can make those decisions for the users and let them concentrate on what they really want to do.
When we don't have real data, we end up feeling that it is best that users decide for themselves, which is not what they paid for. Which, of course, is nothing, so we end up in a "collaborative" design process where the user makes design decisions for us, when using the product, degrading the user experience. The user interface tries to serve everybody equally, and with same kind of interaction for all kinds of users. One size fits all fits nobody well.
I am only just learning myself how to really study and model those needs in a way appropriate for open source.
I just recommended Cooper's The Inmates Are Running the Asylum to Tim last week, since it discusses the question, starting from the roots of the issue, profoundly, still being a relatively short and an entertaining read. I warmly recommend it to anyone taking part in this discussion. (I am not sure but the example above about division of work might have been from that book.)
Apple Human Interface Guidelines: Designing for the 80%
In reply to Olli Savolainen
Re: "Why open source software usability tends to suck" - how do we work around this?
by sam marshall -
About separating jobs - I think this is taken too far. I'm a believer in the artisan-developer model (although it's not for everyone) rather than the production-line-developer model.
In other words I think some programmers are also good at interface design - and if the same person does both, you can do a more efficient job. Interface design is a pretty core part of development, so programmers are quite likely to have this skill. My personal opinion is that all programmers should have it.
The same applies, but to a lesser extent, to other required aspects: graphic design (not so many programmers are good at that) and editing, i.e. writing quality English text (ditto). I don't think all programmers 'should' be good at graphic design or English, but it's certainly convenient if they are.
This is not the same situation as the division between programming and testing. Unlike all of the above, programming and testing are actually contradictory to each other - even if you have good testing skills as well as good programming skills, if you wrote something, that automatically makes you less good at testing it.
On the other hand I absolutely agree about the 'feature requests vs requirements' issue - nobody should develop what people say they want. Think about it and figure it out first.
In Moodle, though, IMO a lot of feature requests do correspond to the underlying requirement - the biggest problem is that Moodle is used (a) by thousands of institutions around the world, (b) from primary schools to universities [and private training too], (c) from organisations who wouldn't know a PHP file if it sat on them, to those with large development teams. These different users are pretty much bound to have lots of different requirements.
--sam
In other words I think some programmers are also good at interface design - and if the same person does both, you can do a more efficient job. Interface design is a pretty core part of development, so programmers are quite likely to have this skill. My personal opinion is that all programmers should have it.
The same applies, but to a lesser extent, to other required aspects: graphic design (not so many programmers are good at that) and editing, i.e. writing quality English text (ditto). I don't think all programmers 'should' be good at graphic design or English, but it's certainly convenient if they are.
This is not the same situation as the division between programming and testing. Unlike all of the above, programming and testing are actually contradictory to each other - even if you have good testing skills as well as good programming skills, if you wrote something, that automatically makes you less good at testing it.
On the other hand I absolutely agree about the 'feature requests vs requirements' issue - nobody should develop what people say they want. Think about it and figure it out first.
In Moodle, though, IMO a lot of feature requests do correspond to the underlying requirement - the biggest problem is that Moodle is used (a) by thousands of institutions around the world, (b) from primary schools to universities [and private training too], (c) from organisations who wouldn't know a PHP file if it sat on them, to those with large development teams. These different users are pretty much bound to have lots of different requirements.
--sam
In reply to sam marshall
Re: "Why open source software usability tends to suck" - how do we work around this?
by Matt Bury -
Hi,
Going back to the referenced article, which I mostly agree with, there's this statement:
Many hackers assume that whatever Microsoft or Apple do is good design, when this is frequently not the case. In imitating the designs of these companies, volunteer projects repeat their mistakes, and ensure that they can never have a better design than the proprietary alternatives.
I'm assuming that by "good design" the author means easy to use. I'd disagree that copying commonly used UI elements and layouts is bad design in any case. I think good UI design is more to do with making it intuitive for the majority of users. If everyone puts the NEXT button at the bottom right of the page, then that's where everyone looks for it regardless of whether it's better to put it at the top right.
Another case that supports this argument is the qwerty keyboard layout, originally invented to slow typists down to stop the keys from jamming on old mechanical typewriters. Subsequent attempts to replace qwerty with more ergonomic, efficient designs have, so far, been unsuccessful.
Personally, I look at Amazon, Yahoo! and Google for UI design because I know that they put a lot of research and resources into it.
From reading the discussion here I think the main problem that we're trying to express was given a name I heard many years ago - "creeping featurism", where software gets new features added ad infinitum until it bloats and then we end up with the 10-90 scenario, where users use 10% of the features 90% of the time.
Going back to the referenced article, which I mostly agree with, there's this statement:
Many hackers assume that whatever Microsoft or Apple do is good design, when this is frequently not the case. In imitating the designs of these companies, volunteer projects repeat their mistakes, and ensure that they can never have a better design than the proprietary alternatives.
I'm assuming that by "good design" the author means easy to use. I'd disagree that copying commonly used UI elements and layouts is bad design in any case. I think good UI design is more to do with making it intuitive for the majority of users. If everyone puts the NEXT button at the bottom right of the page, then that's where everyone looks for it regardless of whether it's better to put it at the top right.
Another case that supports this argument is the qwerty keyboard layout, originally invented to slow typists down to stop the keys from jamming on old mechanical typewriters. Subsequent attempts to replace qwerty with more ergonomic, efficient designs have, so far, been unsuccessful.
Personally, I look at Amazon, Yahoo! and Google for UI design because I know that they put a lot of research and resources into it.
From reading the discussion here I think the main problem that we're trying to express was given a name I heard many years ago - "creeping featurism", where software gets new features added ad infinitum until it bloats and then we end up with the 10-90 scenario, where users use 10% of the features 90% of the time.
In reply to Matt Bury
Re: "Why open source software usability tends to suck" - how do we work around this?
by Olli Savolainen -
Now that you mentioned Google, it intrigues me just how data-driven design should then be,m at the end of the day:
Goodbye, Google
Discussion: Bowman leaves Google
Despite all the opinions here, I still tend to think that using a lot of analysis tools is stil better, while you still use intuition at the end of the day. I have seen both really good and really bad UIs from google.
Goodbye, Google
Discussion: Bowman leaves Google
Despite all the opinions here, I still tend to think that using a lot of analysis tools is stil better, while you still use intuition at the end of the day. I have seen both really good and really bad UIs from google.
"There is a bit of religion here, so be fore-warned. In my temple, I want design to acknowledge the power of soul. My atheist interpretation of said soul is "connectedness".
What I see in the Google-way is dispassionate and thus souless. Is it successful? can't deny they have success. But is the success b/c of the design, or because of something else. IMHO, data-driven design can lead to success, but it is not the type of success I can live with. I.e. I wouldn't want to work for Phillip Morris or Exxon Mobile either. Success without soul is a choice for many.
As a designer though, it seems that soul-lessness is anti-thetical to the artistic roots of design. So if you want to research and derive inspiration from research, or research and live by the data, that is a choice, but I would argue that one is design and the other is not."
"Dave, I think I agree. The problem with data is it requires analysis, which implies interpretation, which can introduce bias anyways. Then you're misled into thinking you're making right decisions based on data when you're actually making it based on subjective interpretation. Using data as an input into design is great. Being tethered to it (or needing it for every decision) is not so good, IMO, FWIW. : )"
In reply to sam marshall
Re: "Why open source software usability tends to suck" - how do we work around this?
by Olli Savolainen -
As long as there are sufficient people doing actual user research and getting the actual knowledge about users to do the design, and to do the usability testing, it is fine by me if we get really skillful jack-of-all-trades programmers to design the UI.
My experience is though that the skill sets do not usually happen in the same person. I am not good at creating a good software architecture. Then again, a crucial property for a UI designer is to be able to sympathize with the users. When you have to go as deep into the intricacies of the constraints presented by the environment you program has to work in, typically you don't have much energy left to sympatize the needs of the users. For one, I could not, when I was also implementing my designs, not nearly as much as I would really have needed to. And that is exactly what you need to be capable of doing, day in and day out.
But in a sense I agree: it is often stated that teams need to be multidisciplinary. I have no experience of the ins and outs of this so I can't tell. But at least the designers also have to understand the constraints the developers are facing, and be capable of discussing them, balancing the pros and cons.
What I was looking for when I said we need people really devoted to the interaction design, I might have been reaching out for something mentioned in this presentation:
The three core attributes. The three questions [critical to any successful UI design team]:
That's the team I want to work on.
(I wish someone would transcribe the essence of that audio recording, since the echo is quite bad)
My experience is though that the skill sets do not usually happen in the same person. I am not good at creating a good software architecture. Then again, a crucial property for a UI designer is to be able to sympathize with the users. When you have to go as deep into the intricacies of the constraints presented by the environment you program has to work in, typically you don't have much energy left to sympatize the needs of the users. For one, I could not, when I was also implementing my designs, not nearly as much as I would really have needed to. And that is exactly what you need to be capable of doing, day in and day out.
But in a sense I agree: it is often stated that teams need to be multidisciplinary. I have no experience of the ins and outs of this so I can't tell. But at least the designers also have to understand the constraints the developers are facing, and be capable of discussing them, balancing the pros and cons.
What I was looking for when I said we need people really devoted to the interaction design, I might have been reaching out for something mentioned in this presentation:
The three core attributes. The three questions [critical to any successful UI design team]:
- Vision "Can everyone on the team describe the experience of using your design five years [from] now?"
- Feedback "In the last six weeks, have you spent more than two hours watching someone use yours or a comptetitor's design?"
- Culture "In the last six weeks, have you rewarded a team member for creating a major design failure?" - throw parties for a team member who has seriously effed up, to provide situations to talk about why this failure was such a good learning opportunity about our client. "Good judgement comes from experience, and experience comes from bad judgement"
That's the team I want to work on.
(I wish someone would transcribe the essence of that audio recording, since the echo is quite bad)
In reply to sam marshall
Re: "Why open source software usability tends to suck" - how do we work around this?
by Alan Trick -
About separating jobs - I think this is taken too far. I'm a believer in the artisan-developer model (although it's not for everyone) rather than the production-line-developer model.
I agree. That was one thing that really bothered be about The Inmates Are Running the Asylum. In my actual experience, whenever you put together two groups like that who have no knowledge about how each other works, it brings out the worst in both parties. You get software that violates platform HIG six ways from Sunday, and to add add insult to injury, it usually behaves in all sorts of really weird ways because the designers didn't understand how the software actually worked.
Programmers shouldn't need to be design professionals, but they should at least have a basic competance (respective to their domain of course, kernel devs don't need to be great UI designers).
In reply to sam marshall
Re: "Why open source software usability tends to suck" - how do we work around this?
by Olli Savolainen -
Sure, the point is not having a division per se, but making sure the essential skills are present in the team and that everybody has resources to do what they know best.
On the other hand I absolutely agree about the 'feature requests vs requirements' issue - nobody should develop what people say they want. Think about it and figure it out first.
In Moodle, though, IMO a lot of feature requests do correspond to the underlying requirement - the biggest problem is that Moodle is used (a) by thousands of institutions around the world, (b) from primary schools to universities [and private training too], (c) from organisations who wouldn't know a PHP file if it sat on them, to those with large development teams. These different users are pretty much bound to have lots of different requirements.
The thing is, I think talking about corresponding to requirements is problematic in the first place. You need to know actual goals of users acting in given contexts and try to match your design to those. If you do not know the users, you will never make enough sense of the overwhelming number of requirements, to organize the UI in a manner that balances the features so that each user can reach their goals with as much ease as possible.
The user data is there to give meaning to the requirements, to make design possible.
On the other hand I absolutely agree about the 'feature requests vs requirements' issue - nobody should develop what people say they want. Think about it and figure it out first.
In Moodle, though, IMO a lot of feature requests do correspond to the underlying requirement - the biggest problem is that Moodle is used (a) by thousands of institutions around the world, (b) from primary schools to universities [and private training too], (c) from organisations who wouldn't know a PHP file if it sat on them, to those with large development teams. These different users are pretty much bound to have lots of different requirements.
The thing is, I think talking about corresponding to requirements is problematic in the first place. You need to know actual goals of users acting in given contexts and try to match your design to those. If you do not know the users, you will never make enough sense of the overwhelming number of requirements, to organize the UI in a manner that balances the features so that each user can reach their goals with as much ease as possible.
The user data is there to give meaning to the requirements, to make design possible.
In reply to Olli Savolainen
Re: "Why open source software usability tends to suck" - how do we work around this?
by A. T. Wyatt -
Greetings, Olli!
You said "Both the arts of requirements engineering and of user centered design promote the idea that we do not do great design by asking users what they need, but by really understanding their actual goals and being capable of translating those needs into designs. Users only know what they need when they see it done."
I would tend to agree with this. An example is the confusion (at least at my institution) generated by overlapping goals that are addressed both in the assignment interface and in the gradebook. I have seen instructors use moodle for quite some time before they realize that they *can* enter grades in the assignment interface. I find it reasonable to assume that the gradebook gets more and more complex because it attempts to address an increasing number of grading tasks.
I always wondered if I would like it better to preserve a stricter separation between the giving of scores/comments (handled by assignment interface) and the calculating of grades (some aggregate of the collected scores handled by the gradebook itself). I am sure a number of people would say "just put it all in one interface; I don't want to have to go two different places to grade, edit grades, and give feedback." I would also suspect that the whole issue is further complicated by the fact that instructors using a traditional grading workflow (face-to-face classroom) might proceed a little differently than by people doing a totally online class.
It seems to me that if you studied a sample of "average" instructors, you might be able to identify some commonalities in the workflow and use these to inform your design.
atw
You said "Both the arts of requirements engineering and of user centered design promote the idea that we do not do great design by asking users what they need, but by really understanding their actual goals and being capable of translating those needs into designs. Users only know what they need when they see it done."
I would tend to agree with this. An example is the confusion (at least at my institution) generated by overlapping goals that are addressed both in the assignment interface and in the gradebook. I have seen instructors use moodle for quite some time before they realize that they *can* enter grades in the assignment interface. I find it reasonable to assume that the gradebook gets more and more complex because it attempts to address an increasing number of grading tasks.
I always wondered if I would like it better to preserve a stricter separation between the giving of scores/comments (handled by assignment interface) and the calculating of grades (some aggregate of the collected scores handled by the gradebook itself). I am sure a number of people would say "just put it all in one interface; I don't want to have to go two different places to grade, edit grades, and give feedback." I would also suspect that the whole issue is further complicated by the fact that instructors using a traditional grading workflow (face-to-face classroom) might proceed a little differently than by people doing a totally online class.
It seems to me that if you studied a sample of "average" instructors, you might be able to identify some commonalities in the workflow and use these to inform your design.
atw
In reply to A. T. Wyatt
Re: "Why open source software usability tends to suck" - how do we work around this?
by Olli Savolainen -
That indeed seems to me an issue, which would get better solved with seeing the interaction at a higher abstraction level. Personas are supposed to give that, Cooper seems to call that + goals + scenarios = Goal-directed Design.
In reply to Matt Gibson
Re: "Why open source software usability tends to suck" - how do we work around this?
by Olli Savolainen -
While working on my master's thesis, I stumbled across an apparently newer version of the article you referred to, and thought you might be interested.
In reply to Olli Savolainen
Re: "Why open source software usability tends to suck" - how do we work around this?
by Matt Gibson -
Thanks Olli, that version has some really good ideas for making the process more effective.
Reading all the comments above has been really enlightening and I feel like I've learned a huge amount. I think it's clear that although there are many good points made about how Moodle there is some need for polishing of the UI to counter some of the tendency towards the creeping featurism (or feeping creaturism) that Matt pointed to. I suggest something similar to the Ubuntu 100 papercuts project. A specifically usability based bugathon style event that would first identify the dozens of little annoyances, rank them in order of annoyingess and then iron them out. I think it'd be a great way to make 2.0 the slickest Moodle yet.
Longer term, many of the suggestions in the later version of the article might be adopted as part of the Moodle development process, like:
"Promote small-scale user testing techniques that are practical for volunteers. Develop and promote screen capture, video recording, and other software that makes tests easier to run. Encourage developers to trust user test results more than user opinions. And write design guidelines that give advice on the common small problems that user tests won’t catch." (my emphasis - this should avoid issues such as Sam raised)
or
"Projects could have a lead human interface designer, who fields everyone else’s suggestions, and works with the programmers in deciding what is implementable. And more detailed design specifications and guidelines could help prevent programmer-specific foibles."
+1 for Olli as Moodle's UI guru
Reading all the comments above has been really enlightening and I feel like I've learned a huge amount. I think it's clear that although there are many good points made about how Moodle there is some need for polishing of the UI to counter some of the tendency towards the creeping featurism (or feeping creaturism) that Matt pointed to. I suggest something similar to the Ubuntu 100 papercuts project. A specifically usability based bugathon style event that would first identify the dozens of little annoyances, rank them in order of annoyingess and then iron them out. I think it'd be a great way to make 2.0 the slickest Moodle yet.
Longer term, many of the suggestions in the later version of the article might be adopted as part of the Moodle development process, like:
"Promote small-scale user testing techniques that are practical for volunteers. Develop and promote screen capture, video recording, and other software that makes tests easier to run. Encourage developers to trust user test results more than user opinions. And write design guidelines that give advice on the common small problems that user tests won’t catch." (my emphasis - this should avoid issues such as Sam raised)
or
"Projects could have a lead human interface designer, who fields everyone else’s suggestions, and works with the programmers in deciding what is implementable. And more detailed design specifications and guidelines could help prevent programmer-specific foibles."
+1 for Olli as Moodle's UI guru
In reply to Matt Gibson
Re: "Why open source software usability tends to suck" - how do we work around this?
by Angelo Abruzzese -
I have been reading this thread and found it quite interesting to follow, whom am I to chime in suffice to say that well the documentation and the intertwined nature of all the open source projects including moodle and mahara can leave something to be desired.
It would seem that everybody is an expert and any documentation on installs are just perfect when in fact they are the shortfall of this type of project.
I found so many souls out there on the way crying after what seems an eternity of fumbling around only to be pointed towards this or that document or tutorial.
I for one don't mind chasing down a rabbit hole, but there better had be some protien at the end of that hunt, for the most part I have found only old skeletal remains.
Including how Mahara seems to be mixed in with Moodle, so which one should you drop into your server www root directory ?
In reply to Matt Gibson
Re: "Why open source software usability tends to suck" - how do we work around this?
by Olli Savolainen -
Such title should make anyone humble, but thank you! (Read your post earlier but did not remember to respond.)
In reply to Matt Gibson
Re: "Why open source software usability tends to suck" - how do we work around this?
by Paul Ganderton -
Hi,
I hesitate to join such an august and obviously technical discussion but I'd like to add something "from the trenches". I'm a user and end-user trainer as well as an ordinary teacher and Moodler so I tend to see this at the sharp end (especially when someone can't find a feature). I'm not too sure that Open Source is automatically poorly designed but it has the propensity to be because it is often made by developers with a vision and not users with a need. I've posted elsewhere in Moodle forums about the need for consistent language usage, definitions and controls and this is improving in Moodle. (Having said that I've spent the last two days getting colleagues to see how to use less obvious features in Microsoft Word
).
The sheer versatility and complexity of Moodle can operate against it because, as has been noted here, it's trying to work in every possible educational situation. I'm not sure there's a simple solution but at least if everyone is aware then there can be some common design elements.
For my own part I tend to get my students/users deeply involved in Moodle by using surveys and forums to see how they view the experience and how I can tweek it to make it better. Every response is replied to and the courses adjusted accordingly. I also try, within my limitations, to contribute to Moodle testing, commenting on forums etc. Perhaps that's the solution - keep thinking, using and talking.
Thanks for the discussion. I'm following links as time allows.
Kind regards,
Paul
I hesitate to join such an august and obviously technical discussion but I'd like to add something "from the trenches". I'm a user and end-user trainer as well as an ordinary teacher and Moodler so I tend to see this at the sharp end (especially when someone can't find a feature). I'm not too sure that Open Source is automatically poorly designed but it has the propensity to be because it is often made by developers with a vision and not users with a need. I've posted elsewhere in Moodle forums about the need for consistent language usage, definitions and controls and this is improving in Moodle. (Having said that I've spent the last two days getting colleagues to see how to use less obvious features in Microsoft Word
).The sheer versatility and complexity of Moodle can operate against it because, as has been noted here, it's trying to work in every possible educational situation. I'm not sure there's a simple solution but at least if everyone is aware then there can be some common design elements.
For my own part I tend to get my students/users deeply involved in Moodle by using surveys and forums to see how they view the experience and how I can tweek it to make it better. Every response is replied to and the courses adjusted accordingly. I also try, within my limitations, to contribute to Moodle testing, commenting on forums etc. Perhaps that's the solution - keep thinking, using and talking.
Thanks for the discussion. I'm following links as time allows.
Kind regards,
Paul
In reply to Matt Gibson
Re: "Why open source software usability tends to suck" - how do we work around this?
by Howard Miller -
I think the article sets out with an assumption that breaks down quickly if we are talking about Moodle (and I assume we are):
"the vast majority of open-source projects are also volunteer projects"
Simply not the case - it might have been once but it isn't now. For better or worse the vast majority of development in Moodle is done by people paid to do it. This certainly applies to core Moodle. Even many of the outlying bits are now maintained by professional developers who'se employers have (presumably) a commercial or operational reason for doing what they do.
"the vast majority of open-source projects are also volunteer projects"
Simply not the case - it might have been once but it isn't now. For better or worse the vast majority of development in Moodle is done by people paid to do it. This certainly applies to core Moodle. Even many of the outlying bits are now maintained by professional developers who'se employers have (presumably) a commercial or operational reason for doing what they do.
In reply to Matt Gibson
Re: "Why open source software usability tends to suck" - how do we work around this?
by Jez H -
That seems to date from 2002? I don't think Wordpress sucks in terms of usability, nor do a number of other open source products. I could also mention a few commercial applications that are a complete nightmare, so I don't think its an open source thing (not any more at least), just down to the level of complexity of a given application and how well thought through things are.
Wordpress is quite simple, Moodle is very complicated.
Automatic seem to have thought about their interface, Moodle HQ never seemed to give usability much thought.
In reply to Jez H
Re: "Why open source software usability tends to suck" - how do we work around this?
by dawn alderson -
Jez, hi
re: Moodle HQ never seemed to give usability much thought.
they are now!
D
In reply to dawn alderson
Re: "Why open source software usability tends to suck" - how do we work around this?
by Jez H -
Yup but I think that may have something to do with "the new shiny" giving Moodle partners a hard time selling "the old clunky". Its a shame it took that kind of shot accross the bow for HQ to wake up to what users had been trying to tell them for years. Not much point having a community if you dont listen to them is it?
Unfortunately some of those users have already voted with their feet and their dollars.
On the bright side the new shiny is very well marketed but not "all that" when you get into it, Moodle remains a better system IMO and will be all the better for a bit of competition on the usability front.
The futures bright, the futures Orange
In reply to Jez H
Re: "Why open source software usability tends to suck" - how do we work around this?
by Colin Fraser -
Sarcasm aside, Dawn, Moodle GUI has always been stodgy, ugly even, but the behind bits work better than anything else I have ever used. (Yes, I love my Moodle, but I am not blinded to its flaws.) Some systems have a great GUI, but they seriously suck because they do not deliver what they promise. If you have ever used Blackboard, then you know what I mean - especially after they bought out WebCT. Another system that looked really good, and did a few things really well, but had limited functionality at the time was Janison. I understand they went into the higher end of the market and have had some success, but whether that is confined to Australia or is elsewhere, dunno....
The point is that I answered a question on Front Page display of Categories or Courses yesterday...I mean "Come Ahn!" in my best John McEnroe style... Why is the Scroll of Death still an issue? Why is it even allowed to be there in the first place? Isn't that also a usability issue?
It is well past time for a stylist to take over Moodle GUI, and get away from the Nuke and Post-Nuke looking Front Page. That is so ermm... ermmm ... 1999ish.
Also, I recall reading somewhere above, or perhaps in another Forum, that one person thought that Devs should understand basic design issues, well GLWTOB! Understanding them does not equate with the ability to practice them competently.
And that is just the GUI, don't get me started on some other things.....
Also... the video standards are changing rapidly, again. So why is there so much trouble about using some video file types in Moodle. This is just one issue that needs be tweaked, polished, improved. Not easy to keep up with external changes perhaps, but it cannot be ignored - and that is the feeling some people have, and are, projecting.
In reply to Colin Fraser
Re: "Why open source software usability tends to suck" - how do we work around this?
by dawn alderson -
Colin,
Re: Sarcasm aside, Dawn,
Oh there is no need to be so sensitive....really, I remember being asked in these forums how I would teach Greek farmers with Moodle-and the topic-EU legislation about cattle or whatever! And that was amongst a number of other peculiar and what might have been -funny to others stuff-...But quite frankly made little sense to me in an authentic way....What a hoot eh.
Giving out is one thing-taking things on the chin is quite another.
RE: Moodle GUI has always been stodgy, ugly even, but the behind bits work better than anything else I have ever used. Oh Colin do you think this needs re-phrasing...LOLs!
(Yes, I love my Moodle, but I am not blinded to its flaws.) -Well good for you!
Some systems have a great GUI, but they seriously suck because they do not deliver what they promise. If you have ever usedBlackboard,
Yes...I have, and I found Bb efficient enough to do what I wanted it to do to support learners. In fact, let us be clear.....any good teacher can make the most of the tools they have....and work with constraints in order to make the most of the affordances of the kit....I suppose the best system I have used to date...is the one at the OU-as a learner....I would not have engaged with it had it not been to my liking....simple as that really....and no scroll of death too....what a charm!-Forum/group engagement....discussion spaces have nice accessibility-so all in all-on top of it -when you look at it from all angles...and so on.......
WebCT-oh it was the starter for 10 and missed the oppt to be great -...that is all.
Not heard of Janison.
It is well past time for a stylist to take over Moodle GUI, and get away from the Nuke and Post-Nuke looking Front Page. That is so ermm... ermmm ... 1999ish.
As is...static words with pics is the theme of the themes for moodle-no? The latest theme floating around here holds promise due to the drop-downs for the learner-interactivity- being key here.....That said there are some awesome customisations out there....which use dataform/sliders and what not.
Also, I recall reading somewhere above, or perhaps in another Forum, that one person thought that Devs should understand basic design issues, well GLWTOB! Understanding them does not equate with the ability to practice them competently. Knowing about Design issues......that aid best practice for L&T -of course....no brainer.
Also... the video standards are changing rapidly, again. So why is there so much trouble about using some video file types in Moodle. This is just one issue that needs be tweaked, polished, improved. Not easy to keep up with external changes perhaps, but it cannot be ignored - and that is the feeling some people have, and are, projecting. HTML 5/updated CSS version too......maybe.
In reply to dawn alderson
Re: "Why open source software usability tends to suck" - how do we work around this?
by Colin Fraser -
HTML 5/updated CSS version too......maybe.
I agree, but it is not happening fast enough. I think that might be the way in which releases are made. This would be a "new feature" maybe, or another change of direction or whatever. It is taking too long, though, and a "backward compatible" plugin for those people who are unable to continuously update their Moodle for whatever reason, is unlikely to be produced.
Knowing aboutDesign issues......that aid best practice for L&T -of course....no brainer.
Again, I agree, but a master coder may never have much of a sense of style. I am trying not to confuse knowledge with competence.
oh it was the starter for 10
No, I am not familiar with this term... I got the second bit, but not this... makes no sense to me, 10 what?
I found Bb efficient enough to do what I wanted it to do
Your version of BB must have been different than the one we had here. Or maybe the people setting it up and running it here didn't know what they were doing. Or perhaps BB was not configured properly, for whatever reason, it kept falling over - especially when accessing test and exam papers. It did not do what I needed it to do, and in no way was it considered efficient. So your experience was a lot better than 250,00 other people that I know about.
As far as themes are concerned, the work being done now with Bootstrap sounds exciting, and I am sure the themes are going to look a lot better. With HTML5, and a larger role for Javascript, Moodle will be less confined by PHP, but time is the issue. The work has to be done, sure, but it is going to take a while. And are those newer themes going to be backwards compatible as well? I would think probably not.
Moving forward is critical for the longer term health of any product, but the right changes need occur a lot faster than they do now. Everyone is too busy worrying about things like versatility, flexibility, availability of a long line of tools, but usability is only recently an issue. Or perhaps it would be more accurate to say that the importance of usability has been overlooked in favour of flexibility. But usability is a lot more important than would seem to be accredited.
and btw.. don't you just love the way I can murder the English language... with real style...
In reply to Colin Fraser
Re: "Why open source software usability tends to suck" - how do we work around this?
by Geoffrey Rowland -
Re 'Starter for 10'
...and here is a little more context. 3rd. paragraph under Game play:
In reply to Geoffrey Rowland
Re: "Why open source software usability tends to suck" - how do we work around this?
by Colin Fraser -
Ahh that is why I didn't get it, The Young Ones were dropped from Oz TV decades ago, University Challenge never appeared on TV in Oz, while James McEvoy is an interesting actor, we have only seen a few of his movies here, and I can't recall a TV movie by that name. Of course, all that may be because I have always found TV as exciting as watching yacht racing, or grass growing and much prefer big screens and decent sound systems. Besides, I learn so much from my computer, every day there is something new to learn... isn't it wonderful?
Today's lesson... cultural bias, or would that be "assumed cultural knowledge", the latter I am thinking, a good thing or a bad thing? How does it affect anything where people from different cultural backgrounds get together?... have to think about that.
In reply to Colin Fraser
Re: "Why open source software usability tends to suck" - how do we work around this?
by dawn alderson -
Colin-re: I agree, but it is not happening fast enough. I think that might be the way in which releases are made. Right. Well you have answered this issue later on....when you say.... With HTML5, and a larger role for Javascript, Moodle will be less confined by PHP, but time is the issue. So, time allocation needs restructuring-really it does.
a "backward compatible" plugin for those people who are unable to continuously update their Moodle for whatever reason, is unlikely to be produced. Why is this an issue....peeps buy into something for a reason...if they don't know about the importance of updates then.........I don't understand that to be honest.
a master coder may never have much of a sense of style. I am trying not to confuse knowledge with competence. Oh stop it! Doing-first sometimes-and reflection with combining existing knowledge and the new from experience is the stuff of some geniuses across history. It is about the art of timing.
Starter for 10...yep good ole Bamba Gascoigne......then Paxo....but also....this is what my dear mum used to say when I about 9 and was acting up to miching mallecho, she would say right.....starter for 10.....warning......10 is grounded for a few days..which inevitably turned into one morning at the weekend............ J
D
In reply to dawn alderson
Re: "Why open source software usability tends to suck" - how do we work around this?
by Colin Fraser -
I have always known that there are people out there who are just not able to update their Moodles as they would like. For example, where I work, our school's Moodle is hosted by a company that was taken over by Blackboard a couple of years ago. We are on v2.5, and while we used to get regular upgrades, these seem to be happening further apart. We cannot upgrade our Moodle whenever we like, we are at the mercy here of the host. That is an agreement made between the original host company and my employer. It sucks, for me - sure, but I live with it. We may be upgraded during our summer break over January, but maybe not, to v2.8.1, or maybe not, and so on. It is not about the importance of updating the technology, it is about the people who are performing the updates. BTW, it is a large site and hosts a lot of Moodles, at least 165 that I am aware of. (How many are active... well I would think a lot now, but it is still early days for a lot of schools. My Employer has gone over to a large, rapacious, software company to provide them with an email/sms/lms/and anything else they want, environment. This has a SSO, but the firewalls are set so you have to log in three to four or five times before it lets you go. (I figured out why it wasn't letting anyone into the Moodle site at all, but my best guess is that the problem will change soon so we will get frustrated with a Moodle that doesn't work, and switch over to their LMS...as soon as it is ready.)
In reply to Colin Fraser
Re: "Why open source software usability tends to suck" - how do we work around this?
by Marcus Green -
Moodle 2.5 is out of support
https://docs.moodle.org/dev/Moodle_2.5.9_release_notes
So they really should be giving you an update very soon.
In reply to Marcus Green
Re: "Why open source software usability tends to suck" - how do we work around this?
by Daniel Neis Araujo -
Hello, everybody
thanks for your contributions here!
I think we've drifted a bit of the original subject here, so what do you think about append a "[CLOSED]" to the discussion forum and start discussing more pratical things in other threads?
Kind regards,
Daniel
In reply to Daniel Neis Araujo
Re: "Why open source software usability tends to suck" - how do we work around this?
by Michael Haskell -
I think that's a great idea.
In fact, what do you think of this very "practical" idea?