30 Replies Latest reply on Oct 13, 2011 9:36 AM by SUBHASH GARG

    Tell us what you think about PISA

    43443 Master

      Do the PISA outcomes reflect your experience of what is happening today within schools and classrooms?

        • Re: Tell us what you think about PISA

          Keen to know why India which is considered to have some darn smart kids is not included in the assessment. Thanks!

            • Re: Tell us what you think about PISA
              43443 Master

              Thank you for your interest in PISA.  Participation depends on interest by governments. Results for 2 states of India will be available in Dec 2011.

                • Re: Tell us what you think about PISA

                  India probably wasn't included because their overall test scores would have been poor, or below average. They may have some of the brightest, but a large % of the lower class would not have scored well. Same with China, selecting their most higly educated province. Indeed the results are not good for the US on balance, but assuming this covers all USA and other countries can pick and choose (now like India) only their best educated geography we need to be careful in making wholesale comparisons.

                    • Re: Tell us what you think about PISA

                      It depends on how you look at the results. If you only look at the final rankings in the local paper... then you are missing out on an important piece. I would even submit one of THE most important pieces of information that PISA offers. Chapter 2 (?) of the results: Overcoming Social Background that examines equity of learning opportunities/outcomes.


                      This essentially examines the relationship between SES and academic results. Which is VERY interesting. The rankings would be different based on this section. This is maybe why some governments are reluctant to engage fully with PISA. The rigour of this section means that you can't hide realities that you might prefer to ignore. And once it is measured and published, then .... you might be compelled to have to act upon the public results.


                      And even IF some people can - with some validity - argue the sampling protocols and the adaptability and the reliability etc. of the tests - the fact remains that the value of these kinds of international tests and publications is the conversations that start about education systems. Conversations that are often rooted in uninformed judgments that are sometimes unchallenged because of who is making the decisions. Conversations that move to becoming depersonalized as talk about evidence takes over. So that informed professional judgments can take over.



                      Just a thought.

                        • Re: Tell us what you think about PISA
                          71887 Novice

                          Definitely agree with you Snoia: the rankings are not the most interesting part of PISA.


                          For me the main message is that with over 20% drop out rates in some countries we are WASTING huge potential. It is an ethical issue and an efficiency issue - if not a survival issue.....


                          So what ARE we waiting for? Superman?


                          Snoia, or anyone else: what do you reckon we need to DO, now that we KNOW?

                            • Re: Tell us what you think about PISA

                              I guess for me, there is a joint responsibility to react to these findings that are critical to moving forward.

                              Local responsibility needs to be taken. That is a taken-for-granted. But I will focus on the continued responsibility for OECD in this response because this is the starting point.

                              If we consider that the PISA results are like an evaluation, then the dissemination of information should reflect the standards of evaluation. I actually think that OECD does a great job in communicating, and every year, they make the information more and more accessible. So first, hats off to meeting more of the reporting standards in evaluation and helping people make sense of the results.

                              But there is always room for improvement. And I think that your question is really reflecting the evaluation standard U8: Concern for Consequences and Influence Evaluations should promote responsible and adaptive use while                 guarding against unintended negative consequences and misuse.

                              Although there is local capacity to promote and use the results, and there should be flexibility in what pieces receive attention depending on country/city/area values and priorities, there should be overall foci too. There should be overall analysis of concerns that top the priorities list across nations. And to me, that falls under the responsibility of the evaluation organization - OECD.

                              So if they want to promote responsible use of the results, such as making sure that countries look past the front page of rankings, then it is the responsibility of the OECD to use knowledge management mechanisms to highlight those particular pieces. The press will do enough with the raw results. The nuances need to be put forward with links to local efforts etc.

                              I would submit that working with the local country teams for use of results is important.

                              Example, Qatar continue to do poorly even if they are one of the biggest improvers. But they are letting their boys down in science. So what is that about? The science gender gap is important in other countries in the MENA region. Is there a possibility for a regional KM piece focusing on this finding? Especially since Qatar needs to rely on their boys to run the country's natural gas industry sooner rather than later.

                              Example, Ontario is being used as an example for the US. If drop out rates are a real concern across the board. Check to see where they have shown some growth and highlight that work. The retention issue in Ontario was a major initiative in the Ontario Ministry of Education. Does the improvement show in PISA? Can this be used to support reflection on this global issue?

                              I am selecting random examples to illustrate that if there are some key overall trends, then linking them back directly to the countries and regions will help them focus their attention because it is meaningful to them.


                              This is about consequential validity

                              Hope that helps

                              Sonia (not snoia, my typo)

                          • Re: Tell us what you think about PISA
                            New User

                            Bruce, I don't agree with you. Results are routinely correlated with socio-economic status. Given that, the dependence of scores on SES within India would be invaluable information.More likely the government of India did not make any effort to connect with PISA. Let me give you a data point. There was a study comparing the GRE (Graduate Record Examination) scores of graduates of the Indian Institutes of Technology to those of MIT and Caltech (USA's premier technology institutes). The Indian group beat the super-elite American group by 7 to 9 points, on the average. .


                            Of course, these are some of the best and brightest learners in India. THE SAME IS TRUE OF THE SHANGHAI SCHOOL KIDS!

                          • Re: Tell us what you think about PISA

                            It took OECD 3 years to come up with the results on 65 countries.Why it will take another year to come up with India's result?

                              • Re: Tell us what you think about PISA
                                565478 Novice

                                Thank you for your question. Along with eight other participants, the Indian states of Himachal Pradesh and Tamil Nadu implemented the PISA 2009 assessment one year later than the main PISA 2009 implementation. Thus, the results from this delayed implementation will likewise be published one year later than those from the main PISA2009 assessment.

                                • Re: Tell us what you think about PISA
                                  New User

                                  This is nothing new. India is routinely excluded from most international compilations, as is mainland China. Western nations are still waking up to these giant countries, which says something about Western ignorance and arrogance. The Times of London's 2011 Global Higher Education rankings listed IIT Bombay at # 361, below obscure places like the University of Guelph. Now, IITB produced a billiionaire venture capitalist and several CEOs of world-class corporations, and Guelph produced no one notable. Does the Times team think all of India is like Slumdog Millionaire? Could well be. I am not joking.

                            • Re: Tell us what you think about PISA

                              The PISA results give us an insight into the young adults of 2020 - a peek into the future if you will. What motivates them to learn? How will they shape a world to their needs? How do they use the unprecedented knowledge resources they have at their fingertips? Beyond the snapshot of country rankings today, PISA gives us a glimpse of the deeper longitudinal trends shaping tomorrow. Given global demographics, you can hardly leave out India and China so it's good to know that there will be more results on those countries soon.

                              • Re: Tell us what you think about PISA

                                I read the headline figures with interest, and have read a bit on the methodology as described in the technical review (2006).  I was surprised to see New Zealand so high.  I am expat English living here, and I don't know of a single European expat with experience of sending their children to European schools and NZ ones that would give much credence to these results.  The sampling system seems at first glance to be fair, although I wonder how the language rules are interpreted.  I also see that in the report NZ has a high level of exclusions (4.58%)  which could conceivably push the score up ? At a qucik read through I can't see any obvious problems with sampling and data processing but it really does not compare to general experience and perceptions at all ! An interesting point for someone to research further.

                                  • Re: Tell us what you think about PISA

                                    The new national government of NZ will be making a lot of changes, mainly because of the poor assessment standards. So what might have happened is that groups of students have filled in questionnaires or teachers assisted them, or they could take them home for a couple of day's, and so on.

                                    • Re: Tell us what you think about PISA

                                      There are other international studies, TIMSS for instance, with more realistic results.

                                        • Re: Tell us what you think about PISA

                                          Thanks for the leads to TIMMS, and also PIRLS is there.  I wonder, does anyone have any suggestiosn as to why the results differ so much ?  As mentioned earlier I am interested in New Zealand, as we live here now.  The TIMMS/ PERLS studies generally put NZ at around 23, whereas PISA has them 4th.  This is a massive difference, and I'd love to know how come this happened.  They can't both be right !  Do the co-pordinators of the 2 programmes talk to each other about these matters ?

                                            • Re: Tell us what you think about PISA
                                              565478 Novice

                                              There are some fundamental differences between PIRLS, TIMSS and PISA.

                                              • PIRLS carrries out tests in reading literacy according to grade as opposed to age (PISA tests 15-year-olds regardless of grade). The students taking the PIRLS tests are also in 4th grade (therefore around age 10 - the minimum age set at 9 1/2 years).  Appendix C of the following publication http://timss.bc.edu/PDF/P06Framework.pdf provides an explanation of the differences.
                                              • The goals of PISA and PIRLS are different. PISA is testing students at the end of compulsory education to assess how well equipped they are to face the world outside formal education - it is not curriculum based. On the contrary the PIRLS assessment draws heavily on extensive investigations into the school curriculum.
                                              • Similarly TIMSS assess students according to grade [(4th (around age10) and 8th (around age 14) grade] and the tests are inspired by the curriculum.
                                                • Re: Tell us what you think about PISA

                                                  I was also given this paper, "Comparing the Similarities and Differences of PISA 2003 and TIMSS", which you can find easily, which provided further insight.  As you say age is certainly one factor, and content another.  For example Tables 4 & 5 show that algebra plays a far less significant role in PISA than TIMMS (24% to 8%).  It raises a moot point about what the point of these tests is, and I am not sure what the declared aims are.  Presumably as they are comparative they will be used by educational policy makers at some point, in whcih case there needs to be a far greater presentation of the underlying principles I think to explain what the tests are trying to assess.  For example some countries may think algebra to be highly important, otherts less so.  If you adopted practices from a PISA high scoring country to improve algebra then they may not gi

                                                  ve the results that you had in mind.

                                          • Re: Tell us what you think about PISA

                                            It's great that PISA administers these tests. They rank education systems objectively, far more so than the biased governments of some participating countries.


                                            I studied in Bulgaria until Grade 8 and our teachers always told us that the Bulgarian education system was the best in the world, that we were lucky to get free education and that Bulgarians were the smartest people on Earth. Teachers spent more time on brainwashing us (without effect in my case) than they did on teaching. Our schools were in disrepair, the textbooks were ridiculous; in elementary school we were treated like POWs. In order to get in a good school one had to bribe teachers on the admissions committees with thousands of US dollars. As a result of this corruption there is a massive gap between the level of education between rich and poor.


                                            Bulgaria's 46th place out of 65 is well-deserved.

                                            • Re: Tell us what you think about PISA
                                              New User

                                              PISA {Program  for International Student Assessment}  is most probably the most salient indicator of international student assessment.  Pisa also is also involved in other studies and global activities.


                                              Some of the results of the 2009 PISA test for 15 year old students in Math, and Science and Reading....65 countries are in New York Times December 8th...2010


                                              Shanghai, China was first with the highest scores in all three categories: math, science, and reading. France was below average in all three of the categories.  The United States was above average in science (barely), reading and below average math.


                                              My thoughts of PISA is positive.


                                              Source: OECE

                                              • Re: Tell us what you think about PISA

                                                I am by no meens an expert in education, yet having been educated to degree level, I have at least some experience. On top of this I have had two children go through conventional education. I feel that quite often a conventional education does not prepare us for a life of work, after we coplete our education.

                                                We tend to look at a potential job after college or university, but there are so many people having to re-train for another type of job, after only a few short years of employment.

                                                Does this imply that the schools do not guide us in the right direction, or is it more of a matter of changing work conditions,along with the collapse of banking sectors world wide.

                                                I had to switch to another type of work due to health issues, not being able to physically do the work that I was trained to do. Unfortunate, but something that was unexpected. Especially once you pass a certain age. Although there are anti dicrimination laws that cover all manner of issues, including age, and handicapped people, yet after being refused employment so many times, when you know that you have the right qualifications, experience etc. it become obvious that you are being discriminated, but a very difficult thing to prove.

                                                I do not feel bitter about such  things, and probably I would have been tempted to make the same decissions had I been the employer, looking at an aplicant with my conditions.

                                                I had a difficult time trying for all sorts of jobs, and in the end I came to the decission that I had to become self employed, with the internet as a way to make money. Re-training is not really there for a would be entrepreneur, you have to stumble around and try to work out how things are done to become successful online.

                                                After a few years of losing money left, right, and center, trying to find out how to do things, evenyually I began to make a living online. Success has cost a lot, but my point about education not preparing us for work stands, I do feel that with the way things are changing, more people now work from home, the system should be there to guide us, how we should be ready for going on our own, how to set up an online businness.

                                                Because of the lack of real information I have created a website that trys to guide people in the right direction when starting an online business. To get started at the lowest possible costs, this is because it's when your out of work, no income that you have to consider going off on your own, with very little money left to invest in a new venture. To help them avoid the scams that abound on the internet today.

                                                Many thanks for the opportunity to speak my mind...

                                                • Tell us what you think about PISA
                                                  New User

                                                  Dear Cassandra,


                                                  I applaud the efforts of your organization. But I must say that any work that treats city-states like Hong Kong on the same footing as China or India is flawed. All the numbers given cannot be representative of the country in question. It would be more valid to tabulate the results by county/ district/ prefecture in the country concerned. Is this information available in the data set? Also, are any socio-economic parameters of the student group being tested provided? This will help ensure that the results are taken in the proper context.


                                                  Subhash Garg PhD

                                                  (retired professor)