DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This Office action is in response to correspondence received January 14, 2026.
Claims 1 and 20 are amended. Claims 1-8 and 20 are pending and have been examined.
Claim Objections
Claim 20 is objected to because of the following informalities: In the final limitation, learned is misspelled as leaned. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8 and 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim(s) recite(s)
Claims 1 and 20, which are similar in scope, recite:
acquire first learning data as information indicating at least one of weather at a first spot at a plurality of times and congestion information as information regarding congestion, and information indicating at least one of operation schedule information on at least one of operation schedule information on trains and operation information indicative of whether there is a train accident, a normal prediction learned model that indicates a number of people in normal times at the first spot at a certain time when first information indicating at least one of the congestion information and the weather at the first spot at the certain time is inputted thereto, and true value information including true values indicating the numbers of people in normal times at the first spot at a plurality of times and true values indicating the numbers of people in emergency times at the first spot at a plurality of times; generate an emergency prediction learned model that indicates the number of people in emergency times at the first spot at the certain time when the first information is inputted thereto by using the first learning data, the normal prediction learned model, and the true value information, and the at least one of the operation schedule information and the operation information, and determining congestion at the first spot at the certain time from the emergency prediction learned model and the normal prediction learned model when information indicating at least one of operation schedule information on at least one of operation schedule information on trains and operation information indicative of whether there is a train accident is acquired.
Claims 1 and 20 recite a mathematical relationship because the claims are reciting relationships between variables and numbers. Namely information indicating weather is taught as a number (percentage likelihood); number of people at a first spot (a number); and congestion at certain places. The claims further apply models, which under a broadest reasonable interpretation are equations or mathematical relations in themselves, to output predictions of the number of people in emergency times (a number) at a first spot (coordinates are numbers). Alternatively this is a certain method of organizing human activity because this manages interactions between people, predicting how people will behave in certain situations in order to give guidance to people. Therefore, the claims recite either a mathematical relationship or a certain method of organizing human activity.
This judicial exception is not integrated into a practical application. The additional elements amount to instructions to apply the abstract idea to a computer because they are circuitry, which would be taught by a general purpose computer, that performs the abstract idea steps. See MPEP 2106.05(f)(2). This is the case both alone and in combination as in combination there are several circuitry elements (in claim 1) that would be taught by a general purpose computer performing the abstract idea steps.
Claim 1 recites the following additional elements:
A learning device comprising: acquiring circuitry to
learning generation circuitry to
outputting circuitry to
Claim 20 recites the following additional elements:
performed by a learning device,
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the reasoning in the practical application section is carried over: for the same reason that apply it instructions are not a practical application, they are also not significantly more than the abstract idea.
Claims 2-8 further recite detail about learning and re-learning a model and using the data in train stations, and adding new data to the models, which further define the abstract idea of claim 1.
Therefore, claims 1-8 and 20 are rejected under 35 USC 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 2, 6, 7, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tatrai et al., US PGPUB 20220254161 A1 ("Tatrai") in view of Rauch, US PGPUB 20080195257 A1 (“Rauch”).
Per claims 1 and 20, which are similar in scope, Tatrai teaches A learning device comprising: acquiring circuitry to acquire first learning data as information indicating at least one of weather at a first spot at a plurality of times and congestion information as information regarding congestion in par 84: “The system 202 may detect crowd density, flow and mood and the complex interrelationships of these three characteristics, which can then be augmented with 3rd party datasets. This may enhance overall accuracy, expanding the ability to utilize leading edge technology and including other data sets in addition to density, flow and mood, e.g. weather, time of day, event type.” See par 085. Weather is an input (which is first learning data) and crowd density which teaches congestion information as information regarding congestion.
Then Tatrai teaches a normal prediction learned model that indicates a number of people in normal times at the first spot at a certain time when first information indicating at least one of the congestion information and the weather at the first spot at the certain time is inputted thereto in par 85: “At step 308, the analysis module 206 predicts crowd information based on the analysis. The crowd information may include emergent crowd characteristics like crowd density, crowd flow, crowd mood, etc., crowd behaviour like positive, negative, or neutral, issues like congestions risks, etc., and one or more actions like deploy more security officials, stop entry of people for some time, etc. The analysis module 206 suggests or predicts one or more actions for managing the plurality of crowds 106A-106N based on the crowd information. Further, the analysis module 206 indicates the one or more issues such as, but not limited to, possible crowd congestion and crowd crush risks based on the speed of change in the crowd density, crowd flow and crowd mood.” Neutral or positive issues teach a normal prediction model and the analysis takes the information that was previously input as taught in par 084.
Then Tatrai teaches and true value information including true values indicating the numbers of people in normal times at the first spot at a plurality of times and true values indicating the numbers of people in emergency times at the first spot at a plurality of times in par 89: “Turning now to FIG. 4, an environment 400 including a system 402 for measuring and managing a plurality of crowds is depicted, in accordance with another embodiment of the present disclosure. A user may access the real-time crowd measurement and management system 402 via a computing device or as a mobile application running on the computing device. As shown, the environment 400 may include a data collection module including a number of data capturing devices 404 like cameras, camera including GPS systems, smart phone, computers, video cameras, etc. The data capturing devices are configured to capture information such as record images, videos, GPS locations, crowd information, and so forth. The data capturing devices 404 may continuously observe and collect first set of crowd data including quantitative data and qualitative data of a plurality of crowds. Each of the plurality of crowds may include a plurality of people. The data collection module may also be configured to receive a second set of crowd data from a plurality of observers present at the plurality of zones via a computing device and a network.” The data collection teaches true value information at the same zones and continuously observe, which teaches plurality of times as for something to be continuous, it must exist in more than one instance of time.
emergency times is taught in par 96: “As density increases and flow decreases over time, crowd mood may begin to deteriorate as crowds become more anxious. A preventative action may be required at a colour zone 502, and at a colour zone 504 to prevent further deterioration of the crowd situation into a colour zone 506, which indicates high risk of crowd crush and congestion.”
Then, Tatrai teaches learning generation circuitry to generate an emergency prediction learned model that outputs the number of people in emergency times at the first spot at the certain time when the first information is inputted thereto by using the first learning data, the normal prediction learned model and the true value information in par 103: “The disclosed real-time crowd measurement and management system is configured to capture crowd data and measure crowd density, crowd flow and crowd mood. The system may use data science to model these characteristics into fitness landscapes. The system then may apply machine learning to predict when the mood of a crowd would turn negative. CNN may be used for person recognition, person count, and density in a crowd. The system may use Bayesian joint probability distribution weighted models for visual mood extraction supplemented by environmental and other factors. The system may be configured for crowd noise monitoring, facial temperatures monitoring, social media scraping key word searches, and so forth. The disclosed system includes multiple camera (existing or supplied) to capture crowd data by observing an activity of crowd etc. at strategic locations. The system may use CNN to identify crowd characteristics. The system is configured to analyze the crowd data based on predictive Bayesian network to predict patterns and changes. The system may provide a dashboard to display insights on crowd behaviour. The system may use predictive capability to continually improve by machine learning module. In some embodiments, the system may use the cameras to automatically capture, identify, and analyze data. Further, the system may efficiently predict the crowd behaviour and potential risks and safety issues with more accuracy.” Measure crowd elements teaches first learning data and true value information, and normal prediction learned model is taught by the predictive Bayesian network, and output number of people in emergency times is taught where the mood of the crowd turns negative and the CNN predicts amount of people. See also par 0107-0108: “"In an example, up to 6 stand-alone camera/computer pods can be deployed to collect and analyze data at any other location. The system uses intelligent camera technology using Convolutional Neural Networks (CNN)' to observe crowd characteristics and measures quantitative and qualitative data. The system may collect information on crowd density, flow and mood of the crowds. The system may identify and anticipate human/crowd behaviour using the data collected. The results are visualized on a dashboard in real-time by using Green, yellow, orange and red indicators for each camera zone denote crowd feeling. The system may include a notification module to send programmed alerts to help to keep decision makers informed, allowing crowd calming actions or changes in design to be implemented that ensure customer satisfaction and event success. The system may use machine learning and artificial intelligence to improves characterization and analysis.
The disclosed system generates one or more reports based on the analysis of crowd data for post-event analysis. The system may enable improved crowd decision making in the real-time using either a combination of latest generation cameras with existing infrastructure or a best of breed change-out to new infrastructure. In some embodiments, strategic upgrade of infrastructure in key locations may enable improved quality, additional customer experience insights and analytics. This system may also enhance an existing CCTV system."
Tatrai does not teach and information indicating at least one of operation schedule information on at least one of operation schedule information on trains and operation information indicative of whether there is a train accident ; using the at least one of the operation schedule information and the operation information; and determining congestion at the first spot at the certain time from the emergency prediction learned model and the normal prediction learned model when information indicating at least one of operation schedule information on at least one of operation schedule information on trains and operation information indicative of whether there is a train accident is acquired.
Rauch teaches passenger counting and security monitoring for trains. See abstract.
Rauch teaches and information indicating at least one of operation schedule information on at least one of operation schedule information on trains and operation information indicative of whether there is a train accident; using the at least one of the operation schedule information and the operation information in par 061 where operation schedule information on trains; is taught: ““The system can be included in the controlling of automatic train systems; it can be designed so that it engages into the train control arrangement or transport means control arrangement, particularly "dispatching", i.e. the controlling of the train and transport means use. Included in this are the train use with regard to length of the trains and size of the transport means units, the frequency, the lines and directions traveled and the destinations of the trains. It can serve to control the acceleration or deceleration of the transport means. "Trains" are to be understood to mean overground or underground rail vehicles such as underground trains or trams, but the invention can equally be used in connection with buses and other vehicles. It can also serve to facilitate the guidance of people and to improve security e.g. in airports.” Because the system is connected to dispatch, operation information on trains traveling through the station is taught. See also for further teaching of train and dispatch, par 125: “If an increased crowd of passengers is detected boarding in an area of a train, the passengers can be guided by the elements of the guidance system to the other, less occupied doors. The system can register the increased influx of passengers and can automatically cause more trains to be provided; in addition, the increased stream of people can already be guided in the approaches to the area, so that a large crowd can be avoided. As the passengers are preferably already detected so early (i.e. for instance already at the station entrance or concourse) that some minutes still elapse until they are standing ready to board at the platform, more trains can be provided in this time, so that the higher number of people can be transported out from the station as quickly as possible. The passengers are informed here that additional trains are being put on, so that they refrain from taking absolutely the first train. An optional possibility consists in connecting escalators, lifts etc. so that the briefly increased stream of people in one direction can be taken away without disturbance. This means that at each escalator for the inflow or outflow from the platform, briefly both are connected for access to the platform, when the system detects that several people are walking towards the platform, whereas no or only a few people are leaving the platform.” Platforms teach station because a station is comprised of train platforms. Using operation schedule information is taught in par 0125 because the schedule information is used to add trains to the schedule.
Rauch then teaches and determining congestion at the first spot at the certain time from the emergency prediction learned model and the normal prediction learned model when information indicating at least one of operation schedule information on at least one of operation schedule information on trains and operation information indicative of whether there is a train accident is acquired in par 0127: “When used to improve rescue operations: When disasters occur, for instance fires, the system can prevent people from running inadvertently into the hazard zone, through the use of the elements of the guidance system, i.e. warning instructions on illuminated signs, acoustic announcements, mechanical blocking-off of dangerous areas etc. The stream of people can then be guided systematically, so that as far as possible no congestion occurs and a sufficient distance from furnaces is maintained. The rescue services are provided with additional information, for example whether people are still situated in a particular area. If a specific transmitter which is not moving is located within a hazard area, it can be concluded that this is a helpless person and steps can be taken. It is preferred that a check can be carried out in a fire incident, as to whether everyone has left the building or are situated in safe areas.” See also par 032: “This makes it possible, particularly in the case of large events with a great influx of people, but also in the case of disasters such as fires etc., to guide streams of people in a desired manner, in order to avoid congestion in specific areas, to break up an existing crowd on a platform as quickly as possible, etc. In this way, platforms can be cleared more quickly, additional trains which are put on by the system in accordance with the invention can already deal with the next influx of people. If the transport operator uses additional trains on different sections of track, people can be directed systematically to these sections.” Disasters include train accidents Rauch is teaching metro stations and fires which under a broadest reasonable interpretation teaches accident.
It would have been obvious to one ordinarily skilled in the art before the effective filing date of the claimed invention to modify the crowd estimating, prediction, and warning tool of Tatrai with the train station information teaching of Rauch because one would be motivated to modify as Tatrai teaches managing crowds in train stations, see par 0128 and Rauch’s teaching in par 002 is that bottlenecks can occur at predicted crowding times and therefore a further optimization can be achieved if crowd and train data is used efficiently. As this would further the ability of Tatrai to reduce or manage crowds in stations, one would be motivated to modify Tatrai with Rauch.
Per claim 2, Tatrai and Rauch teach the limitations of claim 1, above. Tatrai does not teach wherein in a case where the first spot is a station, the first learning data includes operation information on trains traveling through the station.
Examiner notes that the case language makes the claim contingent on the first spot being a station, is a contingent limitation. See MPEP 2111.04(II). Therefore only the structure need be taught for a system claim and under a broadest reasonable interpretation the step need not be taught. The following teaches more than structure but only structure is required.
Rauch teaches wherein in a case where the first spot is a station, the first learning data includes operation information on trains traveling through the station in par 061: “The system can be included in the controlling of automatic train systems; it can be designed so that it engages into the train control arrangement or transport means control arrangement, particularly "dispatching", i.e. the controlling of the train and transport means use. Included in this are the train use with regard to length of the trains and size of the transport means units, the frequency, the lines and directions traveled and the destinations of the trains. It can serve to control the acceleration or deceleration of the transport means. "Trains" are to be understood to mean overground or underground rail vehicles such as underground trains or trams, but the invention can equally be used in connection with buses and other vehicles. It can also serve to facilitate the guidance of people and to improve security e.g. in airports.” Because the system is connected to dispatch, operation information on trains traveling through the station is taught. See also for further teaching of train and dispatch, par 125: “If an increased crowd of passengers is detected boarding in an area of a train, the passengers can be guided by the elements of the guidance system to the other, less occupied doors. The system can register the increased influx of passengers and can automatically cause more trains to be provided; in addition, the increased stream of people can already be guided in the approaches to the area, so that a large crowd can be avoided. As the passengers are preferably already detected so early (i.e. for instance already at the station entrance or concourse) that some minutes still elapse until they are standing ready to board at the platform, more trains can be provided in this time, so that the higher number of people can be transported out from the station as quickly as possible. The passengers are informed here that additional trains are being put on, so that they refrain from taking absolutely the first train. An optional possibility consists in connecting escalators, lifts etc. so that the briefly increased stream of people in one direction can be taken away without disturbance. This means that at each escalator for the inflow or outflow from the platform, briefly both are connected for access to the platform, when the system detects that several people are walking towards the platform, whereas no or only a few people are leaving the platform.” Platforms teach station because a station is comprised of train platforms.
It would have been obvious to one ordinarily skilled in the art before the effective filing date of the claimed invention to modify the crowd estimating, prediction, and warning tool of Tatrai with the train station information teaching of Rauch because one would be motivated to modify as Tatrai teaches managing crowds in train stations, see par 0128 and Rauch’s teaching in par 002 is that bottlenecks can occur at predicted crowding times and therefore a further optimization can be achieved if crowd and train data is used efficiently. As this would further the ability of Tatrai to reduce or manage crowds in stations, one would be motivated to modify Tatrai with Rauch.
Per claim 6, Tatrai and Rauch teach the limitations of claim 1, above. Tatrai further teaches wherein the learning generation circuitry generates a first judgment learned model that outputs information indicating whether the first information is data in normal times or data in emergency times when the first information is inputted thereto or a first judgment learned model that outputs a probability that the first information is data in normal times and a probability that the first information is data in emergency times when the first information is inputted thereto by using the first learning data in par 092: “The analysis module may determine the one or more patterns and changes in mood of the plurality of crowds based on predictive Bayesian network. The analysis module then may predict crowd information comprising at least one of one or more emergent crowd characteristics, an emergent crowd behaviour of the plurality of crowds, and one or more issues based on the analysis in real-time and suggest one or more actions for managing the plurality of crowds based on the crowd information. The analysis module may determine the one or more patterns and changes in mood of the plurality of crowds based on predictive Bayesian network or any other suitable method. The analysis module may indicate possible crowd congestion and crowd crush risks based on the speed of change in the crowd density, crowd flow and crowd mood.”
Per claim 7, Tatrai and Rauch teach the limitations of claim 1, above. Tatrai further teaches and the learning generation circuitry generates a first judgment learned model that outputs information indicating whether information including the first information and the operation information is data in normal times or data in emergency times when the first information and the operation information are inputted thereto or a first judgment learned model that outputs a probability that the information including the first information and the operation information is data in normal times and a probability that the information including the first information and the operation information is data in emergency times when the first information and the operation information are inputted thereto by using the first learning data in par 092: “The analysis module may determine the one or more patterns and changes in mood of the plurality of crowds based on predictive Bayesian network. The analysis module then may predict crowd information comprising at least one of one or more emergent crowd characteristics, an emergent crowd behaviour of the plurality of crowds, and one or more issues based on the analysis in real-time and suggest one or more actions for managing the plurality of crowds based on the crowd information. The analysis module may determine the one or more patterns and changes in mood of the plurality of crowds based on predictive Bayesian network or any other suitable method. The analysis module may indicate possible crowd congestion and crowd crush risks based on the speed of change in the crowd density, crowd flow and crowd mood.”
Tatrai does not teach wherein In a case where the first spot is a station, the first learning data includes operation information on trains traveling through the station.
Rauch teaches wherein in a case where the first spot is a station, the first learning data includes operation information on trains traveling through the station in par 061: “The system can be included in the controlling of automatic train systems; it can be designed so that it engages into the train control arrangement or transport means control arrangement, particularly "dispatching", i.e. the controlling of the train and transport means use. Included in this are the train use with regard to length of the trains and size of the transport means units, the frequency, the lines and directions traveled and the destinations of the trains. It can serve to control the acceleration or deceleration of the transport means. "Trains" are to be understood to mean overground or underground rail vehicles such as underground trains or trams, but the invention can equally be used in connection with buses and other vehicles. It can also serve to facilitate the guidance of people and to improve security e.g. in airports.” Because the system is connected to dispatch, operation information on trains traveling through the station is taught. See also for further teaching of train and dispatch, par 125: “If an increased crowd of passengers is detected boarding in an area of a train, the passengers can be guided by the elements of the guidance system to the other, less occupied doors. The system can register the increased influx of passengers and can automatically cause more trains to be provided; in addition, the increased stream of people can already be guided in the approaches to the area, so that a large crowd can be avoided. As the passengers are preferably already detected so early (i.e. for instance already at the station entrance or concourse) that some minutes still elapse until they are standing ready to board at the platform, more trains can be provided in this time, so that the higher number of people can be transported out from the station as quickly as possible. The passengers are informed here that additional trains are being put on, so that they refrain from taking absolutely the first train. An optional possibility consists in connecting escalators, lifts etc. so that the briefly increased stream of people in one direction can be taken away without disturbance. This means that at each escalator for the inflow or outflow from the platform, briefly both are connected for access to the platform, when the system detects that several people are walking towards the platform, whereas no or only a few people are leaving the platform.” Platforms teach station because a station is comprised of train platforms.
It would have been obvious to one ordinarily skilled in the art before the effective filing date of the claimed invention to modify the crowd estimating, prediction, and warning tool of Tatrai with the train station information teaching of Rauch because one would be motivated to modify as Tatrai teaches managing crowds in train stations, see par 0128 and Rauch’s teaching in par 002 is that bottlenecks can occur at predicted crowding times and therefore a further optimization can be achieved if crowd and train data is used efficiently. As this would further the ability of Tatrai to reduce or manage crowds in stations, one would be motivated to modify Tatrai with Rauch.
Claim(s) 3 and 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tatrai et al., US PGPUB 20220254161 A1 ("Tatrai"), in view of Rauch, US PGPUB 20080195257 A1 (“Rauch”), further in view of Sindagi et al. “A Survey of Recent Advances in CNN-based Single Image Crowd Counting and Density Estimation, Pattern Recognition Letters,” published 2017, available at: < https://engineering.jhu.edu/vpatel36/wp-content/uploads/2018/08/CrowdSurvey_v4.pdf > (“Sindagi”).
Per claim 3, Tatrai and Rauch teach the limitations of claim 1, above. Tatrai further teaches wherein the learning generation circuitry calculates the numbers of people at the first spot at a plurality of times as a plurality of predictive values by using the first learning data, or learning information including the first learning data, and the normal prediction learned model, in par 090: “By using artificial intelligence and convolutional neural networks (CNN), an analysis module present in the cloud-based data management system 402 may identify a plurality of crowd characteristics from the first set of crowd data and the second set of crowd data. The crowd characteristics may include such as, but not limited to, one or more quantitative crowd characteristics and one or more qualitative crowd characteristics. The crowd characteristics may include at least one of a crowd density, a crowd flow, and a crowd mood. Further, the one or more quantitative crowd characteristics may include a number of people in density (ppsqm i.e. people per square metre), the crowd movement of people (ppmpm i.e. people per metre per minute) i.e. speed and direction of people movement(s), and a mood score from negative, negative-neutral, neutral, neutral-positive to positive.”
Tatrai then teaches emergency times and emergency prediction learned model in par 107: “The results are visualized on a dashboard in real-time by using Green, yellow, orange and red indicators for each camera zone denote crowd feeling. The system may include a notification module to send programmed alerts to help to keep decision makers informed, allowing crowd calming actions or changes in design to be implemented that ensure customer satisfaction and event success. The system may use machine learning and artificial intelligence to improves characterization and analysis.” The output from the normal prediction learned model is green or yellow and the output from the emergency prediction learned model is orange or red. The emergency times are the orange and red indicators.
Tatrai does not teach calculates a plurality of errors based on the true values indicating the numbers of people in normal times at the first spot at the plurality of times and the plurality of predictive values, calculates a proportion of each of the plurality of errors in all the errors,
increases the number of pieces of data in… times included in the first learning data or the learning information depending on the proportion, generates the first learning data or the learning information,
in which the number of pieces of data in … times has been increased, as new learning data, and generates the … model by using the true values indicating the numbers of people in … times at the first spot at the plurality of times and the new learning data .
Sindagi teaches techniques to use CNN to count crowds. See abstract.
Sindagi teaches calculates a plurality of errors based on the true values indicating the numbers of people in normal times at the first spot at the plurality of times and the plurality of predictive values, calculates a proportion of each of the plurality of errors in all the errors,
increases the number of pieces of data in… times included in the first learning data or the learning information depending on the proportion, generates the first learning data or the learning information,
in which the number of pieces of data in … times has been increased, as new learning data, and generates the … model by using the true values indicating the numbers of people in … times at the first spot at the plurality of times and the new learning data in page 7: “Inspired by the success of cross-scene crowd counting [107], Walach andWolf [97] performed layered boosting and selective sampling. Layered boosting involves iteratively adding CNN layers to the model such that every new layer is trained to estimate the residual error of the earlier prediction. For instance,
after the first CNN layer is trained, the second CNN layer is trained on the difference between the estimation and ground truth. This layered boosting approach is based on the notion of Gradient Boosting Machines (GBM) [32] which are a subset of powerful ensemble techniques. An overview of their boosting approach is presented in Fig. 5. The other contribution made by the authors is the use of sample selection algorithm
to improve the training process by reducing the effect of low quality samples such as trivial samples or outliers. According to the authors, the samples that are correctly classified early on are trivial samples. Presenting such samples for training even
after the networks have learned to classify them tends to introduce bias in the network for such samples, thereby affecting its generalization performance. Another source of training inefficiency is the presence of outliers such as mislabeled samples. Apart from affecting the network’s performance, these samples increase the training time. To overcome this issue, such samples are eliminated out of the training process for a number of epochs. The authors demonstrated that their method reduces
the count estimation error by 20% to 30% over existing state of-the-art methods at that time on different datasets.” See also Fig 5 on page 7. True count is taught by ground truth. Difference between estimation and ground truth is the error. The increase in data sets are the boosting of select samples, which are new learning data.
It would have been obvious to one ordinarily skilled in the art before the effective filing date of the claimed invention to modify the learning model prediction teaching of Tatrai with the error, true value, and new learning set teaching of Sindagi because Sindagi teaches in page 7 that the error count is reducted 20-30 percent on different datasets using this approach. As this would result in fewer errors in predicting crowd count one would be motivated to modify Tatrai with Sindagi.
Per claim 4, Tatrai, Rauch and Sindagi teach the limitations of claim 3, above. Tatrai further teaches emergency prediction learned model in par 107: “The results are visualized on a dashboard in real-time by using Green, yellow, orange and red indicators for each camera zone denote crowd feeling. The system may include a notification module to send programmed alerts to help to keep decision makers informed, allowing crowd calming actions or changes in design to be implemented that ensure customer satisfaction and event success. The system may use machine learning and artificial intelligence to improves characterization and analysis.” The output from the normal prediction learned model is green or yellow and the output from the emergency prediction learned model is orange or red. The emergency times are the orange and red indicators.
Tatrai does not teach wherein the learning generation circuitry calculates the number of people at the first spot at a certain time by using data in normal times included in the first learning data or the learning information and the normal prediction learned model,
calculates an error between the true value indicating the number of people in normal times at the first spot and the calculated number of people,
and relearns the normal prediction learned model when the calculated error is included in a predetermined range, and the outputting circuitry outputs the relearned normal prediction learned model and the…model
Sindagi teaches the learning generation circuitry calculates the number of people at the first spot at a certain time by using data in normal times included in the first learning data or the learning information and the normal prediction learned model,
calculates an error between the true value indicating the number of people in normal times at the first spot and the calculated number of people,
and relearns the normal prediction learned model when the calculated error is included in a predetermined range, and the outputting circuitry outputs the relearned normal prediction learned model and the…model in teaches in page 7: “Inspired by the success of cross-scene crowd counting [107], Walach andWolf [97] performed layered boosting and selective sampling. Layered boosting involves iteratively adding CNN layers to the model such that every new layer is trained to estimate the residual error of the earlier prediction. For instance, after the first CNN layer is trained, the second CNN layer is trained on the difference between the estimation and ground truth. This layered boosting approach is based on the notion of Gradient Boosting Machines (GBM) [32] which are a subset of powerful ensemble techniques. An overview of their boosting approach is presented in Fig. 5. The other contribution made by the authors is the use of sample selection algorithm to improve the training process by reducing the effect of low quality samples such as trivial samples or outliers. According to the authors, the samples that are correctly classified early on are trivial samples. Presenting such samples for training even after the networks have learned to classify them tends to introduce bias in the network for such samples, thereby affecting its generalization performance. Another source of training inefficiency is the presence of outliers such as mislabeled samples. Apart from affecting the network’s performance, these samples increase the training time. To overcome this issue, such samples are eliminated out of the training process for a number of epochs. The authors demonstrated that their method reduces the count estimation error by 20% to 30% over existing state of-the-art methods at that time on different datasets.” See also Fig 5 on page 7. True count is taught by ground truth. Each layer estimating the previous estimate of error teaches calculated error is included in a predetermined range, which is then relearned with a the new models.
It would have been obvious to one ordinarily skilled in the art before the effective filing date of the claimed invention to modify the learning model prediction teaching of Tatrai with the error, true value, and new learning set teaching of Sindagi because Sindagi teaches in page 7 that the error count is reducted 20-30 percent on different datasets using this approach. As this would result in fewer errors in predicting crowd count one would be motivated to modify Tatrai with Sindagi.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tatrai et al., US PGPUB 20220254161 A1 ("Tatrai"), in view of Rauch, US PGPUB 20080195257 A1 (“Rauch”), further in view of Wang et al., “Deep People Counting in Extremely Dense Crowds,” [online] published 2015, available at: < https://yangliang.github.io/pdf/sp055u.pdf > (“Wang”).
Per claim 5, Tatrai and Rauch teach the limitations of claim 1, above. Tatrai further teaches wherein the learning generation circuitry identifies data in emergency times out of the first learning data by using the first learning data, the normal prediction learned model, and the true values indicating the numbers of people in normal times at the first spot at the plurality of times in par 092: “The analysis module may determine the one or more patterns and changes in mood of the plurality of crowds based on predictive Bayesian network. The analysis module then may predict crowd information comprising at least one of one or more emergent crowd characteristics, an emergent crowd behaviour of the plurality of crowds, and one or more issues based on the analysis in real-time and suggest one or more actions for managing the plurality of crowds based on the crowd information. The analysis module may determine the one or more patterns and changes in mood of the plurality of crowds based on predictive Bayesian network or any other suitable method. The analysis module may indicate possible crowd congestion and crowd crush risks based on the speed of change in the crowd density, crowd flow and crowd mood.” Indicating crowd crush and congestion teaches identifying data in emergency times using the techniques which previously taught normal prediction learned model, true values, and first learning data, see claim 1.
Tatrai does not teach generates the identified data in emergency times as new learning data, and generates the emergency prediction learned model by using the new learning data and the true values indicating the numbers of people in emergency times at the first spot at the plurality of times
Wang teaches people counting in extremely dense crowds for anomaly warning, See page 1.
Wang teaches generates the identified data in emergency times as new learning data, and generates the emergency prediction learned model by using the new learning data and the true values indicating the numbers of people in emergency times at the first spot at the plurality of times in page 2 where data collection and preparation is generating identified data in emergency times (dense crowds) as new learning data and then in pages 3 and 4 (“Experiments”) the new learning data and ground truths (true values) are used to evaluate the emergency times at the first spot at the plurality of times.
It would have been obvious to one ordinarily skilled in the art before the effective filing date of the claimed invention to modify the learning model prediction teaching of Tatrai with the emergency times training teaching of Wang because Wang teaches that by estimating crowds correctly, page 1, tragic stampedes could be predicted and then avoided. As Wang’s teachings directly help to reduce the impact of an emergency, one would be motivated to combine Tatrai with Wang.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tatrai et al., US PGPUB 20220254161 A1 ("Tatrai") in view of Rauch, US PGPUB 20080195257 A1 (“Rauch”) further in view of Frank et al., US PGPUB 20160170998 A1 (“Frank”).
Per claim 8, Tatrai and Rauch teach the limitations of claim 1, above. Tatrai further teaches wherein the acquiring circuitry acquires … learning data as information indicating at least one of the congestion information and the weather at a spot at a plurality of times, and the learning generation circuitry generates a … judgment learned model that outputs information indicating whether information indicating at least one of the congestion information and the weather at a spot at a certain time is data in normal times or data in emergency times when the information is inputted thereto or a second judgment learned model that outputs a probability that the second information is data in normal times and a probability that the information is data in emergency times when the information is inputted thereto by using the learning data in par 84: “The system 202 may detect crowd density, flow and mood and the complex interrelationships of these three characteristics, which can then be augmented with 3rd party datasets. This may enhance overall accuracy, expanding the ability to utilize leading edge technology and including other data sets in addition to density, flow and mood, e.g. weather, time of day, event type.” See par 085. Weather is an input (which is first learning data) and crowd density which teaches congestion information as information regarding congestion.
in par 85: “At step 308, the analysis module 206 predicts crowd information based on the analysis. The crowd information may include emergent crowd characteristics like crowd density, crowd flow, crowd mood, etc., crowd behaviour like positive, negative, or neutral, issues like congestions risks, etc., and one or more actions like deploy more security officials, stop entry of people for some time, etc. The analysis module 206 suggests or predicts one or more actions for managing the plurality of crowds 106A-106N based on the crowd information. Further, the analysis module 206 indicates the one or more issues such as, but not limited to, possible crowd congestion and crowd crush risks based on the speed of change in the crowd density, crowd flow and crowd mood.” Neutral or positive issues teach a normal prediction model and the analysis takes the information that was previously input as taught in par 084.
in par 103: “The disclosed real-time crowd measurement and management system is configured to capture crowd data and measure crowd density, crowd flow and crowd mood. The system may use data science to model these characteristics into fitness landscapes. The system then may apply machine learning to predict when the mood of a crowd would turn negative. CNN may be used for person recognition, person count, and density in a crowd. The system may use Bayesian joint probability distribution weighted models for visual mood extraction supplemented by environmental and other factors. The system may be configured for crowd noise monitoring, facial temperatures monitoring, social media scraping key word searches, and so forth. The disclosed system includes multiple camera (existing or supplied) to capture crowd data by observing an activity of crowd etc. at strategic locations. The system may use CNN to identify crowd characteristics. The system is configured to analyze the crowd data based on predictive Bayesian network to predict patterns and changes. The system may provide a dashboard to display insights on crowd behaviour. The system may use predictive capability to continually improve by machine learning module. In some embodiments, the system may use the cameras to automatically capture, identify, and analyze data. Further, the system may efficiently predict the crowd behaviour and potential risks and safety issues with more accuracy.” Measure crowd elements teaches first learning data and true value information, and normal prediction learned model is taught by the predictive Bayesian network, and output number of people in emergency times is taught where the mood of the crowd turns negative and the CNN predicts amount of people. See also par 0107-0108: “"In an example, up to 6 stand-alone camera/computer pods can be deployed to collect and analyze data at any other location. The system uses intelligent camera technology using Convolutional Neural Networks (CNN)' to observe crowd characteristics and measures quantitative and qualitative data. The system may collect information on crowd density, flow and mood of the crowds. The system may identify and anticipate human/crowd behaviour using the data collected. The results are visualized on a dashboard in real-time by using Green, yellow, orange and red indicators for each camera zone denote crowd feeling. The system may include a notification module to send programmed alerts to help to keep decision makers informed, allowing crowd calming actions or changes in design to be implemented that ensure customer satisfaction and event success. The system may use machine learning and artificial intelligence to improves characterization and analysis.
The disclosed system generates one or more reports based on the analysis of crowd data for post-event analysis. The system may enable improved crowd decision making in the real-time using either a combination of latest generation cameras with existing infrastructure or a best of breed change-out to new infrastructure. In some embodiments, strategic upgrade of infrastructure in key locations may enable improved quality, additional customer experience insights and analytics. This system may also enhance an existing CCTV system."
in par 107: “The results are visualized on a dashboard in real-time by using Green, yellow, orange and red indicators for each camera zone denote crowd feeling. The system may include a notification module to send programmed alerts to help to keep decision makers informed, allowing crowd calming actions or changes in design to be implemented that ensure customer satisfaction and event success. The system may use machine learning and artificial intelligence to improves characterization and analysis.” The output from the normal prediction learned model is green or yellow and the output from the emergency prediction learned model is orange or red.
Tatrai does not teach a spot other than the first spot, a second judgment learned model, second information, second learning data.
Frank teaches generation of crowd based results, see abstract.
Frank teaches a spot other than the first spot, a second judgment learned model, second information, second learning data in par 577-582: “In the illustrated embodiment, the collection module 120 is configured to receive the measurements 501, which in this embodiment comprise measurements corresponding to events in which users were at a first location or a second location. The dynamic scoring module 180 computes scores 569a for the first location and scores 569b for the second location. When computing a score for a certain location from among the first and second locations, the dynamic scoring module 180 utilizes a subset of the measurements 501 comprising measurements of users who were at the certain location, and the measurements in the subset are taken at a time that is after a certain period before a time t, but is not after the time t. Such a score may be referred to as “corresponding to the time t and to the certain location”. Optionally, the certain period is shorter than at least one of the following durations: one minute, ten minutes, one hour, four hours, twelve hours, one day, one week, one month, and one year.
In one embodiment, the dynamic scoring module 180 computes at least the following scores:
a score S.sub.1 corresponding to a time t.sub.1 and to the first location;
a score S.sub.2 corresponding to a time t.sub.2 and to the second location;
a score S.sub.3 corresponding to a time t.sub.3 and to the first location; and
a score S.sub.4 corresponding to a time t.sub.4 and to the second location.
See also par 0588: “In one example, this decision may be made based on the facts that (i) the projected score S.sub.5, which corresponds to the first location is greater than the projected score S.sub.6, which corresponds to the second location, and (ii) prior to the times corresponding to S.sub.5 and S.sub.6, the case was the opposite (i.e., scores for the second location were higher than scores for the first location). Thus, the fact that S.sub.5>S.sub.6 may serve as evidence that in future times after t.sub.5, the scores for the first location are expected to remain higher than the scores for the second location (at least for a certain time). Such a speculation may be based on the fact that the previous scores for those locations indicate that, during the period of time being examined (which includes t.sub.1, . . . , t.sub.4), the scores for the first location increase with the progression of time, while the scores for the second location decrease with the progression of time (see for example, the trends 571a and 571b in FIG. 80b). Thus, in this example, the time t.sub.5 may be the first time for which there is evidence that the scores for the first location are expected to increase above of the scores for the second location, so for times that are after t.sub.5, the recommender module 214 may recommend the first location. For times that are not after t.sub.5, there may be various options. For example, the recommender module 214 may recommend the second location, or recommend both locations the same. It is to be noted that in some embodiments, the time t.sub.5 may serve as the threshold-time t′ mentioned below.” The recommender module teaches a second judgment learned model.
It would have been obvious to one ordinarily skilled in the art before the effective filing date of the claimed invention to modify the learning prediction model teaching of Tatrai with the a spot other than the first spot, a second judgment learned model, second information, second learning data teaching of Frank because one would be motivated to measure multiple locations in order that if a second location was preferable to the first location, then by combining Frank with Tatrai people could move from the first to the second location. This would alleviate the issues with the first location or help people to avoid the issue with the first location. For these reasons one would be motivated to modify Tatrai with Frank.
Therefore, claims 1-8 and 20 are rejected under 35 USC 103.
Response to Remarks
Drawings and Abstract
The drawings and abstract appear correct.
35 USC 101
The amendments further describe the abstract idea and are not a practical application because they are not additional elements. They are further elements that define mathematical relationships or certain methods of organizing human activity. Applicant’s arguments do not detail why these would instead be additional elements (things, for example like computers or other physical items, that are not a part of the abstract idea of determining crowd sizes and emergency response).
Therefore the 101 is maintained.
35 USC 102/3
The claims being amended further search and consideration is necessary and this renders the arguments moot, as the rejection has changed. The 102 is overcome but there is a 103 rejection at this point.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RICHARD W. CRANDALL whose telephone number is (313)446-6562. The examiner can normally be reached M - F, 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anita Coupe can be reached at (571) 270-3614. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RICHARD W. CRANDALL/ Primary Examiner, Art Unit 3619