Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 6, 8-13 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Blayvas et al. (US PGPUB 20190005387) in view of Ishikawa (US PGPUB 200900169168).
[Claim 1]
Blayvas teaches a video recording system, comprising:
a camera (Paragraph 46);
a memory (15) device for storing at least one script code (Paragraph 45);
a processor (10) electrically connected to the camera and the memory device for performing operations when executing the at least one script code (Paragraph 45), the operations comprising:
capturing a plurality of images through the camera to generate a video (Paragraph 40, For example, for object recognition from video, the ANN structure may be divided into blocks, cross-trained for certain conditions and for detection of certain types of objects);
identifying at least one generic object in a plurality of frames in the video (Paragraph 53, For example, object recognition engine 26 analyses image data received from image sensor 20 and detects and/or recognizes in the image data objects and/or other visual properties, such as illumination and/or visibility conditions);
determining an image scene according to the at least one generic object (Paragraph 54, For example, high level engine 14 receives information about detected objects, depth and/or sounds from recognition engines 12, and calculates a current environment state, for example a current map of objects and/or properties of the environment, based on the received information. High level analysis engine 14 may calculate a state of an environment by combining information from the various recognition engine, such as further recognition and/or identification of objects detected by object recognition engine 26, based on information generated by 3D recognition engine 28 and/or vocal recognition engine 30. For example, in case system 100 is implemented in an autonomic vehicle, high level analysis engine 14 may analyze the road situation, taking into account, for example, other vehicles, obstacles, traffic signs, illumination, visibility conditions and/or any other information that may be generated by recognition engines 12. Paragraph 56, For example, in case high level analysis engine 14 recognizes winter and/or snow conditions);
performing a specific object detection according to the image scene to determine whether at least one specific object or at least one specific event appears in the frames (Paragraph 56, For example, in case high level analysis engine 14 recognizes winter and/or snow conditions, attention engine 13 may instruct recognition engines 12, for example, to focus on and/or amplify sensitivity of detection of cars in harsh weather conditions or specifically snow conditions and/or pedestrians in warm winter clothes).; and
Blayvas fails to teach attaching a label to at least one of the frames with the at least one specific object or the at least one specific event and storing the frames with at least one label. However Ishikawa teaches Storage device 120 comprises object detected frame storage means 121 and conversion rule storage unit 122. Object detected frame storage means 121 includes frame storage unit 121a for storing a video frame in which an object appears, and frame-associated information storage unit 121b for storing a frame number of a video frame in which an object appears on an object-by-object basis (Paragraph 82, figs. 2 and 3).
Therefore taking the combined teachings of Blayvas and Ishikawa, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have attached a label to at least one of the frames with the at least one specific object and storing the frames in order to easily recognize persons and places which appear in each frame based on the additional information to calculate an appearance ratio in which each appearing person and place appear in a frame.
[Claim 2]
Blayvas teaches a storage device electrically connected to the processor (Paragraph 42) and configured to store (Paragraph 42, The computer program product may include a tangible non-transitory computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present iso invention) a first neural network (225) and a plurality of second neural networks (240-250), wherein the processor is further for performing the operations comprising:
performing a generic object detection on the frames of the video to identify the at least one generic object in the frames by the first neural network (Paragraph 60, For example, controller 270 may select one of network regions 225, 230, and 235, for example each relating to a different weather class and/or a visibility condition class);
determining the image scene according to the at least one generic object (Paragraph 60, For example, controller 270 may select one of network regions 225, 230, and 235, for example each relating to a different weather class and/or a visibility condition class);
selecting one of the second neural networks according to the image scene ( and one of network regions 240, 245, and 250, for example each relating to different classes of pedestrians, for example grown up people, children or old people); and
performing the specific object detection on the frames to identify the at least one specific object or the at least one specific event in the frames by the one of the second neural networks (Paragraph 60, Thus, controller 270 and/or attention engine 13 may configure classification engine 201 to classify by relevant combinations of network regions, such as the combination of a current identified weather condition and an expected type of pedestrians in a current environment. Accordingly, some embodiments of the present invention provide an adaptive neural network that can be tuned according properties of a current environment).
[Claim 6]
This is a method claim corresponding to apparatus claim 1 and is analyzed and rejected based upon apparatus claim 1.
[Claim 8]
Blayvas teaches determining the image scene according to a category of the at least one generic object, wherein the image scene is one of a road scene, a shopping scene, a travel scene, and a conference scene Paragraph 54, For example, high level engine 14 receives information about detected objects, depth and/or sounds from recognition engines 12, and calculates a current environment state, for example a current map of objects and/or properties of the environment, based on the received information. High level analysis engine 14 may calculate a state of an environment by combining information from the various recognition engine, such as further recognition and/or identification of objects detected by object recognition engine 26, based on information generated by 3D recognition engine 28 and/or vocal recognition engine 30. For example, in case system 100 is implemented in an autonomic vehicle, high level analysis engine 14 may analyze the road situation, taking into account, for example, other vehicles, obstacles, traffic signs, illumination, visibility conditions and/or any other information that may be generated by recognition engines 12. Paragraph 56, For example, in case high level analysis engine 14 recognizes winter and/or snow conditions) but fails to teach a category of the at least one generic object and an area ratio of a bounding box.
[Claim 9]
Blavyas fails to teach when a number of the frames where the at least one specific object appears is 1, storing the frame as an image file; and when a number of the frames where the at least one specific object appears is greater than 1, editing the video into a dynamic image file or a short video file to contain the frames with the at least one specific object.
However Ishikawa teaches When an object is detected (step A05), object detection means 101a stores image information of a frame in which the object is detected in a frame storage unit 121a, and stores the frame number in frame-associated information storage unit 121b on an object-by-object basis (step A06). On the other hand, appearance ratio decision means 101b increments the object detected frame number for measuring the number of frames in which an object is detected on an object-by-object basis, and increments the input video frame number for measuring the total number of frames in the input video by one, respectively (steps A07, A08). (Paragraph 94). Fig. 3 shows that OBJECT A appears at least 3 times, so three frames are stored and if it had appeared in only one frame, then only one frame will be stored.
Therefore taking the combined teachings of Blayvas and Ishikawa, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have when a number of the frames where the at least one specific object appears is 1, storing the frame as an image file; and when a number of the frames where the at least one specific object appears is greater than 1, editing the video into a dynamic image file or a short video file to contain the frames with the at least one specific object in order to create an automatic system to store the appeared detected object in a file depending upon the times of appearance in the frames.
[Claim 10]
Ishikawa teaches that upon receipt of the user's keyboard input, conversion rule selection means 702a selects a conversion rule corresponding to the selected option from conversion rule storage unit 722. Appearance ratio presentation length conversion means 702 calculates the presentation length of each appearing person using the conversion rule selected by conversion rule selection means 702a, and outputs the presentation rules to new video generation means 703. New video presentation means 703 selects frames of each person for the presentation length from frame numbers stored in frame-associated information storage unit 721b on a person-by-person basis, and extracts image information corresponding to the selected frames from frame storage unit 721a. Then the extracted frames are combined to generate a new video which is output to display 730 (Paragraph 182) in order to easily select and display the files to the user upon demand.
[Claim 11]
Blayvas teaches wherein the at least one specific object is one of a traffic sign, a natural landmark, an artificial landmark, a product trademark, a product item, a store name, and a vehicle (Paragraph 49, For example, in a car autonomic navigation system, system 100 needs to be able to detect and/or recognize various possible objects and/or obstacles such as pedestrians, cars, motorcycles, tracks, traffic signs, traffic lights, road lanes, roadside and/or other objects and/or obstacles, and/or conditions such as illumination, visibility and/or road conditions).
[Claim 12]
Blayvas teaches wherein the at least one specific event comprises one of an own traffic behavior, a surrounding traffic behavior, and a movement behavior (Paragraph 55, For example, in a case of an autonomic vehicle system, attention engine 13 may adapt recognition engines 12 to varying road situations. For example, in case high level analysis engine 14 recognizes a ‘children on road’ traffic sign, attention engine 13 may instruct recognition engines 12 to focus on and/or amplify sensitivity of pedestrian detection, and possibly to specifically focus on and/or amplify sensitivity of children pedestrian detection).
[Claims 13 and 15]
These are computer readable claims corresponding to method claims 6 and 9 and are analyzed and rejected based upon method claims 6 and 9.
Claim(s) 3, 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Blayvas et al. (US PGPUB 20190005387), Ishikawa (US PGPUB 200900169168) and in further view of Gaskamp et al. (US PGPUB 20160277688).
[Claim 3]
Blayvas in view of Ishikawa fails to teach a motion sensor electrically connected to the processor and configured to sense a surrounding environment to generate a motion-sensing signal, wherein when the processor receives the motion-sensing signal, the processor controls the camera to start to generate the video. However Gaskamp teaches motion sensor module 404 may be configured to sense motion of animals or other targets (e.g., people) via a PIR sensor 156. The motion sensor module 404 may be configured to generate a motion detect signal upon the PIR sensor 156 receiving reflected light from an animal or such other target within a MFOV of the PIR sensor 156. A motion detect signal may be used to notify or initiate other module(s), for example, a data communications module 406 (for communications-enabled embodiments) to communicate an alert to a user and/or to initiate recording and/or communication of image data/information (Paragraph 59).
Therefore taking the combined teachings of Blayvas, Ishikawa and Gaskamp, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have a motion sensor electrically connected to the processor and configured to sense a surrounding environment to generate a motion-sensing signal, wherein when the processor receives the motion-sensing signal, the processor controls the camera to start to generate the video in order to save power.
[Claim 7]
Blayvas in view of Ishikawa teaches storing the frames with a label (Ishikawa, (Paragraph 82, figs. 2 and 3) but fails to teach transmitting and storing the frames on a cloud server. However Gaskamp teaches a camera unit 100 would operate, for example, in a standby state (ready to detect motion within or about such target area, whether in day or night settings); initiate operation of the camera unit 100 upon detection of such motion; and capture images (whether still or video) for transmission to a remote user via the communication network 1140. The communications network 1140 may include a conventional server 1142 to store and/or manage data transferred through the control and operation of the camera unit 100 and IP network 1144 (or like components as are well known in the art, Paragraph 68).
Therefore taking the combined teachings of Blayvas, Ishikawa and Gaskamp, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have transmitting and storing the frames on a cloud server in order to easily access the images stored remotely without the need for local storage.
[Claim 14]
This is a computer readable claims corresponding to method claim 7 and is analyzed and rejected based upon method claim 7.
Claim(s) 4 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Blayvas et al. (US PGPUB 20190005387), Ishikawa (US PGPUB 200900169168), Gaskamp et al. (US PGPUB 20160277688) and in further view of Yoon (US PGPUB 20230072561).
[Claim 4]
Blayvas in view of Ishikawa fails to teach an acceleration sensor electrically connected to the processor and configured to sense an acceleration information of the camera, wherein the processor controls the camera to stop recording according to the acceleration information and the motion-sensing signal. However Gaskamp teaches process commands module 408 may be configured to receive and process commands for the camera unit 100. For example, commands, such as “enter a low-power mode” (e.g., when there is no detected motion), “initiate image capture,” and “stop image capture.” (Paragraph 61) and A standby module 414 may be configured to cause the camera unit 100 to operate in a “rest” state between periods of activity (e.g., capturing images, transmitting information and data), where many of the electronic components, excluding the PIR sensor 156, are turned off or maintained at low- to very low-power during such rest states. Upon detection of motion within the MFOV, as described above, the standby module 414 is deactivated, and the camera system 100 and the remaining modules, individually or in some combination, are initiated or become active (Paragraph 64) in order to save power.
Blayvas, Ishikawa and Gaskamp fails to teach an acceleration sensor electrically connected to the processor and configured to sense an acceleration information of the camera, wherein the processor controls the camera to stop recording according to the acceleration information. However Yoon teaches In one embodiment, the mini-camera apparatus 16 comprises accelerometers 806 and gyroscopes 807. The accelerometers 806 and gyroscopes 807 may be configured, for example, to create motion-based data (Paragraph 68). In one embodiment, the clip-on mini-camera apparatus 16 comprises a motion detection sensor module 815 and infrared (IR) light source such as LED 816 that detects pre-programmed motion command(s) to initiate and stop the images recording and data acquisition (Paragraph 75).
Therefore taking the combined teachings of Blayvas, Ishikawa, Gaskamp and Yoon, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have an acceleration sensor electrically connected to the processor and configured to sense an acceleration information of the camera, wherein the processor controls the camera to stop recording according to the acceleration information in order to save power.
[Claim 5]
Gaskamp teaches a network electrically connected to the processor and configured to transmit data to a cloud server, wherein the processor transmits the at least one frame with the label to the cloud server according to the acceleration information and the motion-sensing signal (Paragraph 59, A motion sensor module 404 may be configured to sense motion of animals or other targets (e.g., people) via a PIR sensor 156. The motion sensor module 404 may be configured to generate a motion detect signal upon the PIR sensor 156 receiving reflected light from an animal or such other target within a MFOV of the PIR sensor 156. A motion detect signal may be used to notify or initiate other module(s), for example, a data communications module 406 (for communications-enabled embodiments) to communicate an alert to a user and/or to initiate recording and/or communication of image data/information. Paragraph 68, The camera unit 100 would operate, for example, in a standby state (ready to detect motion within or about such target area, whether in day or night settings); initiate operation of the camera unit 100 upon detection of such motion; and capture images (whether still or video) for transmission to a remote user via the communication network 1140. The communications network 1140 may include a conventional server 1142 to store and/or manage data transferred through the control and operation of the camera unit 100 and IP network 1144 (or like components as are well known in the art) in order to easily access the images stored remotely without the need for local storage.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YOGESH K AGGARWAL whose telephone number is (571)272-7360. The examiner can normally be reached Monday - Friday 9:30-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sinh Tran can be reached at 5712727564. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YOGESH K AGGARWAL/Primary Examiner, Art Unit 2637
Prosecution Insights