Fireballs over Japan
- Thread starter Niall
- Start date
Pashalis said:
Agree.
Pashalis said:
It works. Thanks.
Ok, as I see it, there are 43 stations that you need data for (observed meteors by them) for 2013 and years before 2009 (namely 2007 and 2008). I think that the easiest way to ask SonotaCo is to send them the table and ask if they have those numbers that are missing. Any other suggestions?
Pierre said:
To be more precise:
Over the period 2009-2012 the endresult of adding up the resulted counts of all the same cameras, that were active during that time, have recorded a growing number of sky activity which was picked up by those cameras. (about +20% between 2009 and 2012)
Saša said:
Yep, that sounds right to me...
If they have no data from a specific camera or cameras, of this list, in that time frame (2007,2008 and 2013), I'll have to adjust the whole table/chart accordingly, so that we can get fairly accurate statistical data for the whole time frame of 2007-2013.
But that's no problem and can be done quickly.
Saša said:
That's unfortunate... So then we are left with the time frame of 2009-2013 as soon as the data for 2013 comes out... (better then nothing I guess)
Anyways, thanks Saša for contacting them!
Pierre said:
Ok, I'll let you know as soon as I'm finished when the data for 2013 is available...
Maybe there are also other systems for sky activity, like the AMS data, or this Japanese source in place in other countries?
To find this data and maybe also establish a trend for those sources might be interesting...
What about Russia for example, or Canada, Australia etc? Do they have similar data available somewhere?
I've tried to find data for other countries, but with no success as of yet...
Maybe the reason for that, could be that they don't publish their stuff in english, but in their own languages?
Maybe we could start a new thread, asking the whole forum if they know or could search for data regarding sky activity (fireball activity) in other countries?
That would certainly be interesting and we might be able to get fairly accurate data out of those other countries, to get a bigger picture of what is going on in the sky...
To find this data and maybe also establish a trend for those sources might be interesting...
What about Russia for example, or Canada, Australia etc? Do they have similar data available somewhere?
I've tried to find data for other countries, but with no success as of yet...
Maybe the reason for that, could be that they don't publish their stuff in english, but in their own languages?
Maybe we could start a new thread, asking the whole forum if they know or could search for data regarding sky activity (fireball activity) in other countries?
That would certainly be interesting and we might be able to get fairly accurate data out of those other countries, to get a bigger picture of what is going on in the sky...
Found a other fireball network that covers big parts of europe and other non eu countries since 1993:
http://www.imonet.org/
It seems to be the most extensive one as of yet! I'm currently trying to figure out how and if we can get the right data out of it to get a reliable trend... It's a tricky and more complicted one regardind pulling out the right data. Maybe you can help me to figure out more about it?
http://www.imonet.org/
It seems to be the most extensive one as of yet! I'm currently trying to figure out how and if we can get the right data out of it to get a reliable trend... It's a tricky and more complicted one regardind pulling out the right data. Maybe you can help me to figure out more about it?
Pashalis said:
I've looked at it now carefully and I think there is no way I can get reliable statistical data out of the database here:
http://www.imonet.org/database.html
that could show us any kind of reliable trend in regards to fireballs. That is rather unfortunate for such a big video network!
They use a similar system that the Japanese video network is using:
http://www.imo.net/wiki/index.php/Video_Meteor_Observation
Although you can find very detailed information on each camera in their database above (the most detailed yet) for each year and month, it becomes clear to me (from what I've researched as of now) that most if not all of those cameras are just turned on, on specific occasions were they already know that a known meteor shower will occur. In other words most if not all of those cameras do not record all the time and only at night (that is how it looks to me at the moment). Furthermore it seems that those cameras that were active throughout a period until today, target not always the same known showers each year.
That makes it virtually impossible to get reliable statistical data out of this network, for a given time, to see any kind of reliable trend. That is rather sad I've to say...
Actually that is my guess after looking into the data sets of that network carefully, so I could be wrong...
So maybe we should contact them, to see if that is really the case?
Now I'm wondering if not something similar is done with the japanese network? Do they also run their cameras just on specific occasions in the year, of which they know that a meteor shower is expected? Or do they run them all the time?
So I would like to know if those cameras listed in my table for japan, were active all the time, or just turned on at specific time frames within those years?
Saša could you ask SonotaCo if those cameras listed in the table were active all the time, or just at specific occasions within those years?
If some cameras were active all the time, I would need to know which... If some cameras were just active at specific times within those years I would also need further informations about that. I would guess from the data I've gathered from those japanese cameras that they look at the sky much more consistently throughout the years, compared to the imonet network.
So as of now, I would be also careful to trust the trend of my last chart on japan, since we do not know this crucial information...
At this point it looks like the data from AMS might indeed be the "most reliable" as of yet, regarding what is going on in the skies...
Pashalis said:
It looks like they observe only during the night. At least that's explicitly stated for SonotaCo network.
Anyway, I wouldn't dismiss the IMO data yet.
In their database, it can be seen that for period 2006-2012, whole years were covered. I haven't went into details about which camera was operational at which time, what were the weather conditions and what was the number of deployed cameras in particular time period, but after plotting the numbers (overall counts), the rising trend is apparent (see attached preliminary plot). It needs to be determined, though, if the number of operating cameras at certain time has an impact on the trend.
Additionally, on the wiki page it is stated that the array has been rather sparse, but there is no certainty that there was no double counting.
http://www.imonet.org/reports/201311.pdf said:
Pashalis said:
I've sent a mail to SonotaCo asking about that and will reply here when they answer me.
One way to check the data, if somebody wants to do that, would maybe be to select individual camera for given station, plot the overall number of observations using only one camera per station and see what happens to the trend. For example, instead of using all 9 cameras for Nagano 1 station, use only Nagano1_m1, and so on; in that way the double counting would be minimized. It's kind of grueling and tiresome work, the numbers (bar heights) will most probably differ on the case to case basis, but if the trends would be similar for different choices/combinations, it would give more weight to reliability of the data and the applied analysis.
As for the AMS data, there is a human factor involved (in reports), so some kind of bias can probably be expected there as well.
Just my 2 cents.
Saša said:
SonotaCo's reply came.
SonotaCo said:
It's not very helpful, though.
Since there are factors that can't be accounted for, I would suggest to stop worrying about them too much. The selection of the consistent set of cameras over years and looking at their overall counts seems quite reasonable. Yes, the exact individual numbers are probably not 100% reliable, but since the trend is the relevant thing here, I think that Pashalis' analysis method is more than satisfactory regarding the Japanese network data.
Saša said:
If you use the numbers you have used there for the IMO network, the end result is highly falsified and it can't give you any reliable trend. If you look further down on that page you can see all the cameras that were used since they started with their video observations in 1993.
Additionally you can see when the camera was first used and when the last time. If you look further to the right it becomes apparent that even those cameras that are listed as having been active from lets say July 2000 until now, like the camera "TIMES4" have most often only a fraction of Observation nights listed for that time frame. For the camera "TIMES4" it is stated that it has had only 748 Observation nights in almost 13 years...
And if you look at the other cameras that are also listed as having been active until today, most of them also show a rather big disparity between the time frame they are listed as active and the actual Observation nights within this time frame.
In short, it looks like most of them were only sporadically used during those years aka. only when the operater decided to turn it on and not at all time continuously throughout those years. As I've said, that makes the whole data set highly falsified/unreliable for a trend analysis.
I have started to list all those cameras, for the IMO network, that are being listed as having been active till today and quickly realized that they were only active very sporadically. Here you can see the list I've started with those cameras:
https://docs.google.com/spreadsheet/ccc?key=0AosqCCf-0a_XdDdSb0JnckM0c1dzc3d6SHZZcVg0R0E&usp=drive_web#gid=0
The data that is not marked in red, I've looked at carefully, at the database for each camera itself and in which year they were actually active. Those that are marked in red I've not looked into yet, because it became obvious, from the cameras I've looked at, that there is no point in continuing...
Further it became apparent that even those cameras that were active in almost every year, display very varying numbers from year to year, which suggested to me that they were only active at specific times within those years. So I've looked at the "head data" of those cameras and you can see there that most of them (those I've looked at) were only active in some months within those years. Furthermore the same cameras were active in other months in other years!
Pashalis said:
Saša said:
What I would need to know is, which camera was active throughout the whole time frame for both networks (every night) otherwise the data can very easily be falsefied...
But since it seems that nobody is in charge of that information in the japanese network, that the only way to determine that, is to contact each operater which is in charge of each specific camera, if they can let us know this information. It looks to me that we have a good change that many of those 43 cameras I've listed for the japanese network were active all the time, or at least at the same time periods each year, because the numbers for each year seem coherent for that hypothesis. In other words, the numbers for the 43 cameras of the japanese network, are not all over the place in each year, in contrast to the numbers of the IMO network. That suggests to me that those cameras have looked at the sky not so sporadically...
Saša said:
Well I actually think he is right when he says: "So,,, it seems very difficult that the statistics have scientific meaning." unless we can figure out those cameras (in both networks) that were active throughout every night of those years... And I guess the only way to determine that is to ask each operator of a specific camera. Also we would need to know if they always looked at the same part of the sky within that timeframe and if they have changed their lenses/Intensifiers and/or software within that time frame.
In short, a very difficult task, and even slight errors in the data set, can screw up/falsify the whole trend quite significantly...
Pashalis said:
Not exactly.
To "falsify" the rising trend on that plot, there would need to be an increase in overall observation/operation time, either through longer operation time of individual cameras or with using more cameras each year.
Pashalis said:
It's not so important to have all the details if we want to establish the trend.
The thing is that the whole years was covered, and the consistent data with respect to that can be used to see the trend. Not the exact numbers, just the trend. Like I said, to "invent" the rising trend, there should be some deliberate/intentional strongly biased selection applied.
Pashalis said:
Like SonotaCo wrote, some were operating every night and some others maybe not. In their database, it is said
Pashalis said:
Not completely true.
The thing is the data are not perfect. The same is with the AMS data, people report seeing something and that is then checked. Somewhere and sometime, where and when there is less people, consequently there are less reports and the density of the meteors/fireballs is smaller.
We need to deal with what we have, not expecting something perfect that will be piece of cake to analyze.
For me, there is no problem with IMO data, using some cameras for some time, as long as there is not some systematical bias involved. If the cameras were used randomly, according to weather conditions or whatever, it would produce random effect on the overall statistics. The trend can't be faked if there is no systematical bias. The numbers can be affected, but so strong and apparent trend, no.
I think we should focus on that, the trend, not the exact numbers, and check if there is some bias present over the years that could result in the seen trend. For example, to check the overall observational time over years (like it's written in the article Kaigen posted about), not just nights like I did in that plot. And then the number of observed meteors can be plotted versus the observation time for given year. That (the resulting trend) would not be effected with which camera was used, only possible effect would be at what time of the year they were used since at some times there are expected meteor showers and at some other times there are not. I expect that effect on the overall trend would be rather small.
My question is, is there a way to extract the overall observational time for each year (or month; all cameras together), preferably in hours (h)?
If there is, the resulting data can be presented like a 3D plot (or density plot) obs.time - year - No.meteors, or even better option, IMO, would be to plot the density of observed meteors versus year, i.e. No.meteors/obs.time[h] vs. year. In that way the effect of number of used cameras would be accounted for.
I would suggest doing the same for SonotaCo Network as well, if there is a way to extract the information about observational time from their files.
